Jul 10 02:37:51.361250 ip-10-0-139-175 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 02:37:51.361260 ip-10-0-139-175 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 02:37:51.361269 ip-10-0-139-175 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 02:37:51.361672 ip-10-0-139-175 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 02:38:01.481431 ip-10-0-139-175 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 02:38:01.481456 ip-10-0-139-175 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a00054dab8bc4483a179a2ef973fb418 -- Jul 10 02:39:52.099158 ip-10-0-139-175 systemd[1]: Starting Kubernetes Kubelet... Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.542937 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553409 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553427 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553432 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553439 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553443 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553447 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553452 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:39:52.610521 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553456 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553460 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553464 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553468 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553478 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553482 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553486 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553490 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553494 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553499 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553503 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553507 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553511 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553515 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553519 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553523 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553531 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553544 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553549 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553554 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:39:52.611777 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553557 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553562 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553566 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553570 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553586 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553590 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553594 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553599 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553602 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553610 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553614 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553619 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553623 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553627 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553630 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553634 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553638 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553643 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553646 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553650 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:39:52.612616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553654 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553658 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553666 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553669 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553673 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553678 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553682 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553687 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553691 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553695 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553700 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553704 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553708 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553714 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553727 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553731 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553742 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553755 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553759 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:39:52.613450 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553764 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553768 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553771 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553775 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553779 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553783 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553791 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553795 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553801 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553806 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553811 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553816 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553820 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553824 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553828 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553832 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553850 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553854 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553859 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:39:52.614295 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553867 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553871 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553875 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553880 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553884 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553888 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553892 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553904 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553909 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553914 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553918 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553926 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553938 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.553942 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557825 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557878 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557898 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557902 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557906 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557909 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:39:52.615135 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557912 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557916 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557919 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557923 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557926 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557929 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557932 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557936 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557939 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557942 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557945 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557948 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557951 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557954 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557957 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557960 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557962 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557965 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557968 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557970 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:39:52.615988 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557973 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557975 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557979 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557983 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557986 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557990 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557992 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557995 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.557998 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558000 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558003 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558006 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558008 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558011 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558014 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558016 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558018 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558023 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558026 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558028 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:39:52.616539 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558031 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558034 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558036 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558039 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558041 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558044 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558047 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558049 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558052 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558055 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558060 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558066 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558070 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558074 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558077 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558080 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558084 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558087 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558090 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:39:52.617118 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558092 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558095 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558097 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558100 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558103 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558107 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558109 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558112 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558114 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558117 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558121 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558125 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558128 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558131 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558134 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558137 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558140 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558143 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558145 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558148 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:39:52.617642 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558151 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558153 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558156 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558159 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558162 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558165 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558167 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558170 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558173 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558176 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558186 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558190 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558192 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558195 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558651 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558656 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558659 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558664 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558668 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:39:52.618244 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558671 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558674 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558677 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558679 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558682 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558685 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558688 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558690 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558694 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558697 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558699 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558703 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558707 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558710 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558713 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558716 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558718 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558721 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558724 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:39:52.619024 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558726 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558730 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558732 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558735 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558744 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558747 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558749 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558752 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558754 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558758 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558760 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558763 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558765 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558767 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558770 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558773 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558775 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558779 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558782 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558785 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:39:52.619568 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558787 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558791 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558793 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558796 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558798 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558801 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558804 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558806 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558810 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558814 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558820 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558822 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558825 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558829 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558832 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558835 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558855 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558873 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558877 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:39:52.620216 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558880 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558883 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558885 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558888 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558891 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558893 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558896 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558899 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558902 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558904 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558907 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558909 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558912 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558915 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558918 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558921 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558923 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558926 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558928 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558931 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:39:52.620730 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558933 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558936 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558938 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558944 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558946 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558949 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558952 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558955 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558957 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558960 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558962 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558980 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558984 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558987 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558989 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.558992 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559071 2585 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559079 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559100 2585 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559104 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559108 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559112 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 02:39:52.621456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559116 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559120 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559123 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559127 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559131 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559134 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559137 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559140 2585 flags.go:64] FLAG: --cgroup-root="" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559143 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559146 2585 flags.go:64] FLAG: --client-ca-file="" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559149 2585 flags.go:64] FLAG: --cloud-provider="external" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559152 2585 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559159 2585 flags.go:64] FLAG: --cluster-domain="" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559164 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559167 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559171 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559174 2585 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559178 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559181 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559184 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559188 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559191 2585 flags.go:64] FLAG: --contention-profiling="false" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559200 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559203 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 02:39:52.622106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559207 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559209 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559214 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559217 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559220 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559223 2585 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559227 2585 flags.go:64] FLAG: --enable-server="true" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559230 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559237 2585 flags.go:64] FLAG: --event-burst="100" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559241 2585 flags.go:64] FLAG: --event-qps="50" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559244 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559247 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559250 2585 flags.go:64] FLAG: --eviction-hard="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559254 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559257 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559260 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559263 2585 flags.go:64] FLAG: --eviction-soft="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559266 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559269 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559272 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559275 2585 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559279 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559282 2585 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559286 2585 flags.go:64] FLAG: --feature-gates="" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559290 2585 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 02:39:53.769161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559293 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 02:39:53.710245 ip-10-0-139-175 systemd[1]: Started Kubernetes Kubelet. Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559296 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559299 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559302 2585 flags.go:64] FLAG: --healthz-port="10248" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559306 2585 flags.go:64] FLAG: --help="false" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559308 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-139-175.ec2.internal" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559318 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559321 2585 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559324 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559328 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559331 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559334 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559337 2585 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559340 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559343 2585 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559346 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559349 2585 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559352 2585 flags.go:64] FLAG: --kube-reserved="" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559355 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559358 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559361 2585 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559364 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559367 2585 flags.go:64] FLAG: --lock-file="" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559370 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 02:39:53.771024 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559373 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559377 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559383 2585 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559386 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559390 2585 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559393 2585 flags.go:64] FLAG: --logging-format="text" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559396 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559400 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559403 2585 flags.go:64] FLAG: --manifest-url="" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559405 2585 flags.go:64] FLAG: --manifest-url-header="" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559410 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559413 2585 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559417 2585 flags.go:64] FLAG: --max-pods="110" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559420 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559423 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559431 2585 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559435 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559438 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559441 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559444 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559452 2585 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559455 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559458 2585 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 02:39:53.772823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559462 2585 flags.go:64] FLAG: --pod-cidr="" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559464 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559471 2585 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559473 2585 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559477 2585 flags.go:64] FLAG: --pods-per-core="0" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559479 2585 flags.go:64] FLAG: --port="10250" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559482 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559487 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-037595c361eb277c4" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559490 2585 flags.go:64] FLAG: --qos-reserved="" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559493 2585 flags.go:64] FLAG: --read-only-port="10255" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559496 2585 flags.go:64] FLAG: --register-node="true" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559499 2585 flags.go:64] FLAG: --register-with-taints="" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559503 2585 flags.go:64] FLAG: --registry-burst="10" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559505 2585 flags.go:64] FLAG: --registry-qps="5" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559509 2585 flags.go:64] FLAG: --reserved-cpus="" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559513 2585 flags.go:64] FLAG: --reserved-memory="" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559516 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559519 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559522 2585 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559525 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559529 2585 flags.go:64] FLAG: --runonce="false" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559531 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559535 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559537 2585 flags.go:64] FLAG: --seccomp-default="false" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559540 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 02:39:53.774094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559549 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559553 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559556 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559559 2585 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559562 2585 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559565 2585 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559568 2585 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559571 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559574 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559577 2585 flags.go:64] FLAG: --system-cgroups="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559579 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559585 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559587 2585 flags.go:64] FLAG: --tls-cert-file="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559590 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559600 2585 flags.go:64] FLAG: --tls-min-version="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559603 2585 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559605 2585 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559608 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559611 2585 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559614 2585 flags.go:64] FLAG: --v="2" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559618 2585 flags.go:64] FLAG: --version="false" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559624 2585 flags.go:64] FLAG: --vmodule="" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559628 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.559631 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559762 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:39:53.775349 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559766 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559770 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559773 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559776 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559779 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559781 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559784 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559788 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559798 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559801 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559803 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559807 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559809 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559812 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559815 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559818 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559820 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559823 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559825 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:39:53.780344 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559828 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559831 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559834 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559850 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559853 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559855 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559858 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559860 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559863 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559867 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559870 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559872 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559875 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559878 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559881 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559883 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559886 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559888 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559891 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:39:53.781196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559894 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559896 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559899 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559908 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559912 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559914 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559917 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559920 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559922 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559925 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559927 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559930 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559932 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559935 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559937 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559942 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559945 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559948 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559950 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559953 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:39:53.782068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559955 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559958 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559962 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559965 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559967 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559970 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559979 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559981 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559984 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559987 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559989 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559992 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559994 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559997 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.559999 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560002 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560010 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560013 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560016 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560018 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:39:53.783595 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560021 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560023 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560026 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560028 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560032 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560036 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560039 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560043 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560045 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560048 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560050 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560053 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560055 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560059 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560064 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560067 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560069 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560073 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560076 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:39:53.784786 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.560079 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.560708 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.568031 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.568051 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.568657 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.568664 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568730 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568738 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568742 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568745 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568748 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568751 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568753 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568756 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:39:53.785875 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568759 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568761 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568764 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568767 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568769 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568772 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568775 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568778 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568780 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568783 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568785 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568788 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568790 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568793 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568796 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568799 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568802 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568805 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568808 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568810 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:39:53.787921 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568813 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568817 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568819 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568822 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568824 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568827 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568829 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568831 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568834 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568886 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568889 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568892 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568896 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568898 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568901 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568904 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568906 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568909 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568911 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568914 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:39:53.789020 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568916 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568921 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568924 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568927 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568930 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568933 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568936 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568940 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568943 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568945 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568948 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568952 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568954 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568957 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568960 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568963 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568965 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568968 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568970 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:39:53.790317 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568973 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568975 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568979 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568981 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568984 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568987 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568989 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568992 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568994 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568997 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.568999 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569002 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569004 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569007 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569010 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569013 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569015 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569019 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569022 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:39:53.791191 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569024 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569028 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569031 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569034 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569037 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569039 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569042 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569045 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569047 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569050 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569053 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569057 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569060 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.569065 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569205 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 02:39:53.792487 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569210 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569216 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569219 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569221 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569224 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569227 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569230 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569232 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569235 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569238 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569240 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569243 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569247 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569251 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569254 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569257 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569261 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569265 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569269 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 02:39:53.793139 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569272 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569275 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569278 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569280 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569283 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569285 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569288 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569290 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569294 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569297 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569299 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569301 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569304 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569307 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569310 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569313 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569315 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569318 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569321 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569323 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 02:39:53.793770 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569327 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569329 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569332 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569334 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569337 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569340 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569342 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569345 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569347 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569350 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569352 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569357 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569359 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569362 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569364 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569368 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569371 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569374 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569377 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569379 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 02:39:53.794643 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569382 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569385 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569388 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569390 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569393 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569395 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569399 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569401 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569404 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569407 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569409 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569412 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569414 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569417 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569420 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569422 2585 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569425 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569427 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569429 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569432 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 02:39:53.795512 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569434 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569437 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569439 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569443 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569446 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569449 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569451 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569454 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569456 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569459 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569462 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569464 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569467 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569469 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569472 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569475 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569477 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569479 2585 feature_gate.go:328] unrecognized feature gate: Example Jul 10 02:39:53.796370 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:52.569484 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.569489 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.569637 2585 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.571822 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.573260 2585 server.go:1019] "Starting client certificate rotation" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.573365 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.573403 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.577491 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.581557 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.596603 2585 log.go:25] "Validated CRI v1 runtime API" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.600444 2585 log.go:25] "Validated CRI v1 image API" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.601248 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 02:39:53.797020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.603644 2585 fs.go:135] Filesystem UUIDs: map[50ba1197-3c21-44f5-872c-50eb7d3dee4f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 97dac9ec-3d50-4351-a2dc-c8d84f07c92a:/dev/nvme0n1p3] Jul 10 02:39:53.797899 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.603659 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.608520 2585 manager.go:216] Machine: {Timestamp:2026-07-10 02:39:52.607647352 +0000 UTC m=+0.381265744 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100993 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25911122f33fd88ec8941b69d70635 SystemUUID:ec259111-22f3-3fd8-8ec8-941b69d70635 BootID:a00054da-b8bc-4483-a179-a2ef973fb418 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:19:91:45:95:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:19:91:45:95:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:23:99:fc:d2:90 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.608694 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.608825 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.610713 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.610754 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-175.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.612079 2585 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.612095 2585 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 02:39:53.798124 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.612106 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.612123 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.619513 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.623022 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633673 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633819 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633865 2585 kubelet.go:486] "Attempting to sync node with API server" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633876 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633897 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633908 2585 kubelet.go:398] "Adding apiserver pod source" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.633918 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.634368 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.634390 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.634877 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xk59g" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.637104 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639518 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639547 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639686 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639699 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639706 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639712 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639720 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.639726 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.640308 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.640320 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.640333 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.640341 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.640356 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.640369 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.641079 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:52.641095 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.709298 2585 server.go:1268] "Started kubelet" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.709737 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.711394 2585 server.go:310] "Adding debug handlers to kubelet server" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.713299 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.713378 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.713464 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xk59g" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.714038 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.714084 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-175.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 02:39:53.798559 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.714138 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.715329 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.720339 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.720345 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.720474 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.721116 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.721118 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.721136 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.721249 2585 reconstruct.go:97] "Volume reconstruction finished" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.721255 2585 reconciler.go:29] "Reconciler: start to sync state" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.722230 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.722389 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-175.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.722136 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-175.ec2.internal.18c0cd4f7c3e2e05 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-175.ec2.internal,UID:ip-10-0-139-175.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-175.ec2.internal,},FirstTimestamp:2026-07-10 02:39:53.709276677 +0000 UTC m=+0.418085204,LastTimestamp:2026-07-10 02:39:53.709276677 +0000 UTC m=+0.418085204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-175.ec2.internal,}" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727429 2585 factory.go:55] Registering systemd factory Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727459 2585 factory.go:223] Registration of the systemd container factory successfully Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.727574 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727675 2585 factory.go:153] Registering CRI-O factory Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727691 2585 factory.go:223] Registration of the crio container factory successfully Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727745 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727769 2585 factory.go:103] Registering Raw factory Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.727795 2585 manager.go:1196] Started watching for new ooms in manager Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.728259 2585 manager.go:349] Starting recovery of all containers Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.729357 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.737812 2585 manager.go:354] Recovery completed Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.742212 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.744148 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.744169 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.744179 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.744940 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.744949 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.744964 2585 state_mem.go:36] "Initialized new in-memory state store" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.745490 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.746157 2585 policy_none.go:49] "None policy: Start" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.746170 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 02:39:53.799880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.746180 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 02:39:53.801671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.746560 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:53.801671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.746583 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:53.801671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.746595 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:53.801671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.747339 2585 policy_none.go:47] "Start" Jul 10 02:39:53.857792 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.831684 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:53.932899 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.931993 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.940302 2585 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-175.ec2.internal\" not found" node="ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962390 2585 manager.go:337] "Starting Device Plugin manager" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.962425 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962434 2585 server.go:87] "Starting device plugin registration server" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962678 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962688 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962893 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962956 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:53.962962 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.963508 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 02:39:54.026090 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:53.963558 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.063281 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.064264 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.064292 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.064308 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.064335 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.072202 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.072227 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-175.ec2.internal\": node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.096470 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.105057 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.105095 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.105125 2585 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.105163 2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 02:39:54.130359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.107680 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.197288 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.226397 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal"] Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.226478 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.228163 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.228192 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.228203 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.229688 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.229778 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.229805 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.231180 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.231185 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.231208 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.231220 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.231208 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.231283 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.233143 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.233170 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.233935 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientMemory" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.233951 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 02:39:54.258750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.233962 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeHasSufficientPID" Jul 10 02:39:54.349534 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.297452 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.349534 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.311855 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-175.ec2.internal\" not found" node="ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.349534 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.317479 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-175.ec2.internal\" not found" node="ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.349534 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.334213 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddd074d505fbaa3df61bf060561b2fdc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal\" (UID: \"ddd074d505fbaa3df61bf060561b2fdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.349534 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.334237 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb0a5737ac1f577cebb69602803ae7c2-config\") pod \"kube-apiserver-proxy-ip-10-0-139-175.ec2.internal\" (UID: \"fb0a5737ac1f577cebb69602803ae7c2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.349534 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.334266 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ddd074d505fbaa3df61bf060561b2fdc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal\" (UID: \"ddd074d505fbaa3df61bf060561b2fdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.433381 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.398136 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.467281 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.462345 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb0a5737ac1f577cebb69602803ae7c2-config\") pod \"kube-apiserver-proxy-ip-10-0-139-175.ec2.internal\" (UID: \"fb0a5737ac1f577cebb69602803ae7c2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.467392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.467284 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ddd074d505fbaa3df61bf060561b2fdc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal\" (UID: \"ddd074d505fbaa3df61bf060561b2fdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.467392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.467312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddd074d505fbaa3df61bf060561b2fdc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal\" (UID: \"ddd074d505fbaa3df61bf060561b2fdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.467392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.462459 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fb0a5737ac1f577cebb69602803ae7c2-config\") pod \"kube-apiserver-proxy-ip-10-0-139-175.ec2.internal\" (UID: \"fb0a5737ac1f577cebb69602803ae7c2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.467392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.467368 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddd074d505fbaa3df61bf060561b2fdc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal\" (UID: \"ddd074d505fbaa3df61bf060561b2fdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.467392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.467369 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ddd074d505fbaa3df61bf060561b2fdc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal\" (UID: \"ddd074d505fbaa3df61bf060561b2fdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.498543 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.498499 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.599059 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.599020 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.614227 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.614195 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.621017 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.620994 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.637484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.637462 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 02:39:54.637614 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.637596 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 02:39:54.699955 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.699917 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.719795 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.719716 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 02:34:53 +0000 UTC" deadline="2028-03-18 12:04:53.396390952 +0000 UTC" Jul 10 02:39:54.719795 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.719747 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14817h24m58.676647546s" Jul 10 02:39:54.800446 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.800401 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.840794 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.840761 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:39:54.901096 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:54.901065 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-175.ec2.internal\" not found" Jul 10 02:39:54.916845 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.916819 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:39:54.920796 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.920768 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.930426 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.930410 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 02:39:54.933471 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.933456 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" Jul 10 02:39:54.943754 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:54.943735 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 02:39:55.148964 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:55.148929 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd074d505fbaa3df61bf060561b2fdc.slice/crio-880d9a48324bbcd2878cbd0130d7ef87c81f752eb1fa4e569ec84a420b8404ae WatchSource:0}: Error finding container 880d9a48324bbcd2878cbd0130d7ef87c81f752eb1fa4e569ec84a420b8404ae: Status 404 returned error can't find the container with id 880d9a48324bbcd2878cbd0130d7ef87c81f752eb1fa4e569ec84a420b8404ae Jul 10 02:39:55.153513 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:55.153487 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0a5737ac1f577cebb69602803ae7c2.slice/crio-24af07c2fbe42931acbad76a9a7dc40a03fa65ae90611dac490d9823b8935d1b WatchSource:0}: Error finding container 24af07c2fbe42931acbad76a9a7dc40a03fa65ae90611dac490d9823b8935d1b: Status 404 returned error can't find the container with id 24af07c2fbe42931acbad76a9a7dc40a03fa65ae90611dac490d9823b8935d1b Jul 10 02:39:55.154463 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.154443 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 02:39:55.226156 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.226129 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:39:55.700358 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.700327 2585 apiserver.go:52] "Watching apiserver" Jul 10 02:39:55.710661 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.710463 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 02:39:55.711529 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.711491 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-5wkwz","kube-system/konnectivity-agent-857wz","openshift-dns/node-resolver-k6qd5","openshift-image-registry/node-ca-69ntc","openshift-multus/network-metrics-daemon-zn6fq","openshift-network-operator/iptables-alerter-wt5wx","openshift-ovn-kubernetes/ovnkube-node-bvvqn","kube-system/global-pull-secret-syncer-wd6fm","kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq","openshift-cluster-node-tuning-operator/tuned-7jj5p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal","openshift-multus/multus-additional-cni-plugins-2kxtv","openshift-multus/multus-qlcvg"] Jul 10 02:39:55.713560 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.713080 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:55.713560 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.713153 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:39:55.714450 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.714432 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.715844 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.715824 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.717051 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.716821 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rgp28\"" Jul 10 02:39:55.717051 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.716831 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 02:39:55.717051 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.716855 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 02:39:55.717298 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.717256 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.717859 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.717839 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.718271 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.718250 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6mgjg\"" Jul 10 02:39:55.718369 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.718317 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.718735 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.718703 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:55.718824 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.718772 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:39:55.719917 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.719897 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.720010 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.719928 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j7qrb\"" Jul 10 02:39:55.720010 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.719936 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 02:39:55.720129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.720098 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.720694 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.720673 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.720965 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.720929 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 02:39:55.723664 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.723636 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.723762 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.723742 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 02:39:55.724082 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.724065 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z256g\"" Jul 10 02:39:55.724292 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.724276 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.724750 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.724514 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.725129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.725106 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.725218 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.725178 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:39:55.726859 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.726514 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 02:39:55.726859 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.726718 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 02:39:55.726859 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.726817 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.727456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.727335 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q46k4\"" Jul 10 02:39:55.727543 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.727509 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.727775 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.727756 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 02:39:55.727775 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.727768 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 02:39:55.728014 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.727995 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.728469 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.728452 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.729157 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.729140 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.729451 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.729332 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 02:39:55.730910 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.730893 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.731946 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.731927 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.732272 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.732255 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5vwhx\"" Jul 10 02:39:55.732947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.732860 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-64knq\"" Jul 10 02:39:55.732947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.732907 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.733252 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.733236 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.735163 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.735087 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.735523 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.735502 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 02:39:55.735714 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.735696 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 02:39:55.735842 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.735825 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 02:39:55.735973 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.735956 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pnvpt\"" Jul 10 02:39:55.736626 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.736609 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 02:39:55.736825 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.736808 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 02:39:55.737630 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.737613 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 02:39:55.738183 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.738159 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 02:39:55.738260 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.738192 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cxs44\"" Jul 10 02:39:55.771326 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.771299 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lfz77" Jul 10 02:39:55.773981 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.773956 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-etc-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774092 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.773996 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:55.774152 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774092 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb1de517-e278-4495-bc0e-02c07d5964d3-hosts-file\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.774152 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774118 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2bnj\" (UniqueName: \"kubernetes.io/projected/a9cf5549-afa4-46fa-aff4-270bfe266cbe-kube-api-access-p2bnj\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.774152 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774144 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c39ba390-03ea-466c-80c1-8dfdb19e32dd-agent-certs\") pod \"konnectivity-agent-857wz\" (UID: \"c39ba390-03ea-466c-80c1-8dfdb19e32dd\") " pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.774302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774167 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:55.774302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774199 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9cf5549-afa4-46fa-aff4-270bfe266cbe-host-slash\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.774302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774220 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-run-netns\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774256 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfzpf\" (UniqueName: \"kubernetes.io/projected/41878b22-23b8-43fc-b63d-e753fd7c5b5c-kube-api-access-kfzpf\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.774302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774280 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-kubelet\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774301 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-slash\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774327 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-ovn\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-node-log\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774377 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1de517-e278-4495-bc0e-02c07d5964d3-tmp-dir\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-systemd\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774424 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774455 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-cni-bin\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774488 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pks5\" (UniqueName: \"kubernetes.io/projected/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-kube-api-access-7pks5\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774517 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x74p\" (UniqueName: \"kubernetes.io/projected/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-kube-api-access-7x74p\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774545 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9cf5549-afa4-46fa-aff4-270bfe266cbe-iptables-alerter-script\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.774582 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774568 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-log-socket\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774590 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovnkube-script-lib\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774616 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774643 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c39ba390-03ea-466c-80c1-8dfdb19e32dd-konnectivity-ca\") pod \"konnectivity-agent-857wz\" (UID: \"c39ba390-03ea-466c-80c1-8dfdb19e32dd\") " pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774694 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41878b22-23b8-43fc-b63d-e753fd7c5b5c-host\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774725 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-var-lib-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774751 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774805 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbmnn\" (UniqueName: \"kubernetes.io/projected/fb1de517-e278-4495-bc0e-02c07d5964d3-kube-api-access-jbmnn\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774835 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-cni-netd\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774856 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovnkube-config\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774877 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-env-overrides\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774901 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovn-node-metrics-cert\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.774947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774925 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41878b22-23b8-43fc-b63d-e753fd7c5b5c-serviceca\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.775265 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.774957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-systemd-units\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.788230 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.788208 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lfz77" Jul 10 02:39:55.822587 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.822561 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 02:39:55.875454 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875417 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-modprobe-d\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875463 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-netns\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875486 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-kubelet\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875515 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-ovn\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875537 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-node-log\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875566 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-lib-modules\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875587 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875636 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-node-log\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.875642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-ovn\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875687 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-var-lib-kubelet\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875716 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06dbb287-4849-421f-9e0a-d60266e1b45b-cni-binary-copy\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875751 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdfch\" (UniqueName: \"kubernetes.io/projected/dfeee125-4ac5-414e-b804-fe0b83d82ffb-kube-api-access-cdfch\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875804 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-systemd\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875826 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875843 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-socket-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875863 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysconfig\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875885 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-host\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875894 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-systemd\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875904 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-tuned\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875951 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-os-release\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.876022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.875980 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-log-socket\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876619 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876537 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-system-cni-dir\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.876619 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876571 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df6sj\" (UniqueName: \"kubernetes.io/projected/3689d3fd-6681-4c30-aa1f-53d31bae8186-kube-api-access-df6sj\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.876619 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876586 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-log-socket\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876619 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-cni-bin\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876633 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-cni-multus\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876686 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgc8k\" (UniqueName: \"kubernetes.io/projected/06dbb287-4849-421f-9e0a-d60266e1b45b-kube-api-access-xgc8k\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876717 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c39ba390-03ea-466c-80c1-8dfdb19e32dd-konnectivity-ca\") pod \"konnectivity-agent-857wz\" (UID: \"c39ba390-03ea-466c-80c1-8dfdb19e32dd\") " pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41878b22-23b8-43fc-b63d-e753fd7c5b5c-host\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876790 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.876823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876812 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-462zn\" (UniqueName: \"kubernetes.io/projected/73d8f94b-573d-4917-be6c-b3f528d1f167-kube-api-access-462zn\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876804 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-cni-netd\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876891 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-run-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876894 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-env-overrides\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876930 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovn-node-metrics-cert\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.876994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-daemon-config\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877025 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-etc-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877076 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb1de517-e278-4495-bc0e-02c07d5964d3-hosts-file\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877107 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-sys-fs\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.877133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877132 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-kubernetes\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877154 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-run\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877175 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-cni-binary-copy\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877215 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-socket-dir-parent\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877240 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-etc-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877245 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-cni-netd\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877293 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-env-overrides\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877316 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41878b22-23b8-43fc-b63d-e753fd7c5b5c-host\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877350 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c39ba390-03ea-466c-80c1-8dfdb19e32dd-agent-certs\") pod \"konnectivity-agent-857wz\" (UID: \"c39ba390-03ea-466c-80c1-8dfdb19e32dd\") " pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877355 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb1de517-e278-4495-bc0e-02c07d5964d3-hosts-file\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877349 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c39ba390-03ea-466c-80c1-8dfdb19e32dd-konnectivity-ca\") pod \"konnectivity-agent-857wz\" (UID: \"c39ba390-03ea-466c-80c1-8dfdb19e32dd\") " pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877378 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877407 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-cnibin\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877525 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfzpf\" (UniqueName: \"kubernetes.io/projected/41878b22-23b8-43fc-b63d-e753fd7c5b5c-kube-api-access-kfzpf\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877565 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-kubelet\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.877634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877591 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-slash\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877614 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1de517-e278-4495-bc0e-02c07d5964d3-tmp-dir\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.877622 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877638 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877696 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-slash\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877705 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-kubelet\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.877716 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:39:56.37769676 +0000 UTC m=+3.086505289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysctl-conf\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877744 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877782 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-systemd\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877817 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877844 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-cni-bin\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pks5\" (UniqueName: \"kubernetes.io/projected/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-kube-api-access-7pks5\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877890 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.878354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877911 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877921 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1de517-e278-4495-bc0e-02c07d5964d3-tmp-dir\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877931 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-cni-bin\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877938 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d8f94b-573d-4917-be6c-b3f528d1f167-tmp\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.877973 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-multus-certs\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878004 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x74p\" (UniqueName: \"kubernetes.io/projected/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-kube-api-access-7x74p\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878030 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9cf5549-afa4-46fa-aff4-270bfe266cbe-iptables-alerter-script\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878054 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovnkube-script-lib\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878079 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0991902f-cbfb-41d6-9399-519aac9b646b-dbus\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878100 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-os-release\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878126 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-registration-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878158 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-var-lib-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878214 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbmnn\" (UniqueName: \"kubernetes.io/projected/fb1de517-e278-4495-bc0e-02c07d5964d3-kube-api-access-jbmnn\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878234 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-sys\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.879079 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878252 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-system-cni-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878274 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovnkube-config\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878293 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41878b22-23b8-43fc-b63d-e753fd7c5b5c-serviceca\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878312 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-systemd-units\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878332 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0991902f-cbfb-41d6-9399-519aac9b646b-kubelet-config\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878343 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-run-ovn-kubernetes\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878365 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-cnibin\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878410 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-var-lib-openvswitch\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-hostroot\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878504 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-etc-kubernetes\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878536 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2bnj\" (UniqueName: \"kubernetes.io/projected/a9cf5549-afa4-46fa-aff4-270bfe266cbe-kube-api-access-p2bnj\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878569 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-device-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878596 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysctl-d\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-cni-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.879747 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-k8s-cni-cncf-io\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878690 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-conf-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878716 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9cf5549-afa4-46fa-aff4-270bfe266cbe-host-slash\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878738 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-run-netns\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878760 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878774 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9cf5549-afa4-46fa-aff4-270bfe266cbe-iptables-alerter-script\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878842 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovnkube-script-lib\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878869 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-host-run-netns\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878900 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9cf5549-afa4-46fa-aff4-270bfe266cbe-host-slash\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878917 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-systemd-units\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878960 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovnkube-config\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.880400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.878961 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/41878b22-23b8-43fc-b63d-e753fd7c5b5c-serviceca\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.882098 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.882055 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-ovn-node-metrics-cert\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.882296 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.882277 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c39ba390-03ea-466c-80c1-8dfdb19e32dd-agent-certs\") pod \"konnectivity-agent-857wz\" (UID: \"c39ba390-03ea-466c-80c1-8dfdb19e32dd\") " pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:55.884149 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.884128 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:39:55.884149 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.884153 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:39:55.884293 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.884164 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:55.884293 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.884221 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:39:56.384204076 +0000 UTC m=+3.093012616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:55.886847 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.886821 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfzpf\" (UniqueName: \"kubernetes.io/projected/41878b22-23b8-43fc-b63d-e753fd7c5b5c-kube-api-access-kfzpf\") pod \"node-ca-69ntc\" (UID: \"41878b22-23b8-43fc-b63d-e753fd7c5b5c\") " pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:55.888298 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.888255 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2bnj\" (UniqueName: \"kubernetes.io/projected/a9cf5549-afa4-46fa-aff4-270bfe266cbe-kube-api-access-p2bnj\") pod \"iptables-alerter-wt5wx\" (UID: \"a9cf5549-afa4-46fa-aff4-270bfe266cbe\") " pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:55.888298 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.888276 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pks5\" (UniqueName: \"kubernetes.io/projected/9cbc0665-a310-4e2c-97f8-acb6fcdfaae7-kube-api-access-7pks5\") pod \"ovnkube-node-bvvqn\" (UID: \"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7\") " pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:55.888643 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.888625 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x74p\" (UniqueName: \"kubernetes.io/projected/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-kube-api-access-7x74p\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:55.890795 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.890774 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbmnn\" (UniqueName: \"kubernetes.io/projected/fb1de517-e278-4495-bc0e-02c07d5964d3-kube-api-access-jbmnn\") pod \"node-resolver-k6qd5\" (UID: \"fb1de517-e278-4495-bc0e-02c07d5964d3\") " pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-sys-fs\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-kubernetes\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979483 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-run\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979498 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-cni-binary-copy\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-socket-dir-parent\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979522 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-sys-fs\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-cnibin\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979576 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979597 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-cnibin\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979624 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979662 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysctl-conf\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979679 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-systemd\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979697 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-socket-dir-parent\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979698 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-run\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.979841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979698 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-kubernetes\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979793 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d8f94b-573d-4917-be6c-b3f528d1f167-tmp\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979835 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-multus-certs\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979840 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-systemd\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0991902f-cbfb-41d6-9399-519aac9b646b-dbus\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979867 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979834 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysctl-conf\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979883 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-multus-certs\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-os-release\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-registration-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979946 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-sys\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979966 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-system-cni-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.979994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0991902f-cbfb-41d6-9399-519aac9b646b-kubelet-config\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980002 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0991902f-cbfb-41d6-9399-519aac9b646b-dbus\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980007 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-os-release\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980021 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-registration-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980018 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-cnibin\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980046 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-system-cni-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980063 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-hostroot\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980071 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0991902f-cbfb-41d6-9399-519aac9b646b-kubelet-config\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980090 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-etc-kubernetes\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980102 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-hostroot\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980067 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-cnibin\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980093 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-sys\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980124 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-etc-kubernetes\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980126 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-device-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980169 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysctl-d\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.980998 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980175 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-device-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980193 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-cni-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980209 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-cni-binary-copy\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980217 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-k8s-cni-cncf-io\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980256 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-k8s-cni-cncf-io\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980266 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-conf-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980293 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980294 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysctl-d\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980314 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-modprobe-d\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980332 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-conf-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980344 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-netns\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980368 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-run-netns\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980307 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-cni-dir\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980370 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-kubelet\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980400 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-modprobe-d\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980407 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-kubelet\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980418 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-lib-modules\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980443 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-var-lib-kubelet\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980466 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06dbb287-4849-421f-9e0a-d60266e1b45b-cni-binary-copy\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980489 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdfch\" (UniqueName: \"kubernetes.io/projected/dfeee125-4ac5-414e-b804-fe0b83d82ffb-kube-api-access-cdfch\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980513 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-lib-modules\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980516 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980547 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-socket-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980571 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysconfig\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980590 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-host\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980608 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-tuned\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980629 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-os-release\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-system-cni-dir\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.981978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980702 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df6sj\" (UniqueName: \"kubernetes.io/projected/3689d3fd-6681-4c30-aa1f-53d31bae8186-kube-api-access-df6sj\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-cni-bin\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980752 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-cni-multus\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980778 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgc8k\" (UniqueName: \"kubernetes.io/projected/06dbb287-4849-421f-9e0a-d60266e1b45b-kube-api-access-xgc8k\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-462zn\" (UniqueName: \"kubernetes.io/projected/73d8f94b-573d-4917-be6c-b3f528d1f167-kube-api-access-462zn\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.980861 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-daemon-config\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981214 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06dbb287-4849-421f-9e0a-d60266e1b45b-cni-binary-copy\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981231 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-var-lib-kubelet\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981275 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3689d3fd-6681-4c30-aa1f-53d31bae8186-system-cni-dir\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981313 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-cni-multus\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981403 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dfeee125-4ac5-414e-b804-fe0b83d82ffb-socket-dir\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981452 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-sysconfig\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981460 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-host-var-lib-cni-bin\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981494 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/73d8f94b-573d-4917-be6c-b3f528d1f167-host\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.982617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981609 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06dbb287-4849-421f-9e0a-d60266e1b45b-multus-daemon-config\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.983371 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.981777 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:55.983371 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.981816 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06dbb287-4849-421f-9e0a-d60266e1b45b-os-release\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.983371 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:55.981882 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:39:56.481858635 +0000 UTC m=+3.190667151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:55.983371 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.982746 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.983371 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.982763 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3689d3fd-6681-4c30-aa1f-53d31bae8186-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.983371 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.982919 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d8f94b-573d-4917-be6c-b3f528d1f167-tmp\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.984562 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.984542 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/73d8f94b-573d-4917-be6c-b3f528d1f167-etc-tuned\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:55.990895 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.990860 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdfch\" (UniqueName: \"kubernetes.io/projected/dfeee125-4ac5-414e-b804-fe0b83d82ffb-kube-api-access-cdfch\") pod \"aws-ebs-csi-driver-node-dlwvq\" (UID: \"dfeee125-4ac5-414e-b804-fe0b83d82ffb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:55.991026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.990988 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df6sj\" (UniqueName: \"kubernetes.io/projected/3689d3fd-6681-4c30-aa1f-53d31bae8186-kube-api-access-df6sj\") pod \"multus-additional-cni-plugins-2kxtv\" (UID: \"3689d3fd-6681-4c30-aa1f-53d31bae8186\") " pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:55.991443 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.991423 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgc8k\" (UniqueName: \"kubernetes.io/projected/06dbb287-4849-421f-9e0a-d60266e1b45b-kube-api-access-xgc8k\") pod \"multus-qlcvg\" (UID: \"06dbb287-4849-421f-9e0a-d60266e1b45b\") " pod="openshift-multus/multus-qlcvg" Jul 10 02:39:55.991596 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:55.991552 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-462zn\" (UniqueName: \"kubernetes.io/projected/73d8f94b-573d-4917-be6c-b3f528d1f167-kube-api-access-462zn\") pod \"tuned-7jj5p\" (UID: \"73d8f94b-573d-4917-be6c-b3f528d1f167\") " pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:56.028781 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.028752 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-857wz" Jul 10 02:39:56.039368 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.039299 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k6qd5" Jul 10 02:39:56.050295 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.050267 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" Jul 10 02:39:56.055057 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.055035 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-69ntc" Jul 10 02:39:56.063610 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.063589 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wt5wx" Jul 10 02:39:56.073587 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.073567 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:39:56.080329 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.080311 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" Jul 10 02:39:56.088081 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.088064 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" Jul 10 02:39:56.093816 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.093796 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qlcvg" Jul 10 02:39:56.110001 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.109951 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" event={"ID":"ddd074d505fbaa3df61bf060561b2fdc","Type":"ContainerStarted","Data":"880d9a48324bbcd2878cbd0130d7ef87c81f752eb1fa4e569ec84a420b8404ae"} Jul 10 02:39:56.111170 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.111148 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" event={"ID":"fb0a5737ac1f577cebb69602803ae7c2","Type":"ContainerStarted","Data":"24af07c2fbe42931acbad76a9a7dc40a03fa65ae90611dac490d9823b8935d1b"} Jul 10 02:39:56.383857 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.383764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:56.384010 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.383922 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:56.384010 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.383998 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:39:57.383980222 +0000 UTC m=+4.092788758 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:56.485101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.485061 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:56.485300 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.485116 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:56.485300 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.485181 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:56.485300 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.485256 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:39:57.485237083 +0000 UTC m=+4.194045612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:56.485468 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.485299 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:39:56.485468 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.485324 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:39:56.485468 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.485336 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:56.485468 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:56.485394 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:39:57.485377841 +0000 UTC m=+4.194186356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:56.790755 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.790662 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:34:55 +0000 UTC" deadline="2028-02-09 10:22:28.356321695 +0000 UTC" Jul 10 02:39:56.790755 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:56.790710 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13903h42m31.56561873s" Jul 10 02:39:56.966592 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:56.966565 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1de517_e278_4495_bc0e_02c07d5964d3.slice/crio-345d8d3cc7daa04d97fe3f437d31328a292303e09fa9a0d8944fa82b0e79e538 WatchSource:0}: Error finding container 345d8d3cc7daa04d97fe3f437d31328a292303e09fa9a0d8944fa82b0e79e538: Status 404 returned error can't find the container with id 345d8d3cc7daa04d97fe3f437d31328a292303e09fa9a0d8944fa82b0e79e538 Jul 10 02:39:56.968585 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:56.968559 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9cf5549_afa4_46fa_aff4_270bfe266cbe.slice/crio-0509a890fba798509317e0d211d7be1d0ec154030a85b20b195a91e32e7d52a8 WatchSource:0}: Error finding container 0509a890fba798509317e0d211d7be1d0ec154030a85b20b195a91e32e7d52a8: Status 404 returned error can't find the container with id 0509a890fba798509317e0d211d7be1d0ec154030a85b20b195a91e32e7d52a8 Jul 10 02:39:56.969496 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:56.969466 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3689d3fd_6681_4c30_aa1f_53d31bae8186.slice/crio-97bdb5c840053fd33118b2bd52e0d27ddecfd9f50da69970f8e99f44c60fe003 WatchSource:0}: Error finding container 97bdb5c840053fd33118b2bd52e0d27ddecfd9f50da69970f8e99f44c60fe003: Status 404 returned error can't find the container with id 97bdb5c840053fd33118b2bd52e0d27ddecfd9f50da69970f8e99f44c60fe003 Jul 10 02:39:56.976397 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:56.976364 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41878b22_23b8_43fc_b63d_e753fd7c5b5c.slice/crio-0e8675e89add530a9aae68294667dcbc56f260b4b1021e55da58fffcbae6fccb WatchSource:0}: Error finding container 0e8675e89add530a9aae68294667dcbc56f260b4b1021e55da58fffcbae6fccb: Status 404 returned error can't find the container with id 0e8675e89add530a9aae68294667dcbc56f260b4b1021e55da58fffcbae6fccb Jul 10 02:39:56.977068 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:39:56.977039 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc39ba390_03ea_466c_80c1_8dfdb19e32dd.slice/crio-5cb8a60db6f1162da0b928bc8893775725a21afb891c8e750f158508b8b0d139 WatchSource:0}: Error finding container 5cb8a60db6f1162da0b928bc8893775725a21afb891c8e750f158508b8b0d139: Status 404 returned error can't find the container with id 5cb8a60db6f1162da0b928bc8893775725a21afb891c8e750f158508b8b0d139 Jul 10 02:39:57.106420 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.106385 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:57.106420 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.106416 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:57.106620 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.106509 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:39:57.106678 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.106617 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:39:57.115539 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.115505 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" event={"ID":"fb0a5737ac1f577cebb69602803ae7c2","Type":"ContainerStarted","Data":"3d770b33ce7ddc29ad1fb7e6a56b4ad70d49bad148346b7eba6fa225e50eec13"} Jul 10 02:39:57.116629 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.116605 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-857wz" event={"ID":"c39ba390-03ea-466c-80c1-8dfdb19e32dd","Type":"ContainerStarted","Data":"5cb8a60db6f1162da0b928bc8893775725a21afb891c8e750f158508b8b0d139"} Jul 10 02:39:57.117915 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.117888 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-69ntc" event={"ID":"41878b22-23b8-43fc-b63d-e753fd7c5b5c","Type":"ContainerStarted","Data":"0e8675e89add530a9aae68294667dcbc56f260b4b1021e55da58fffcbae6fccb"} Jul 10 02:39:57.120026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.120003 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qlcvg" event={"ID":"06dbb287-4849-421f-9e0a-d60266e1b45b","Type":"ContainerStarted","Data":"5dbfead3da7088ac6f4fc50aa9f52fa2642383cc1341c962c30bb184ce334053"} Jul 10 02:39:57.120966 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.120946 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" event={"ID":"dfeee125-4ac5-414e-b804-fe0b83d82ffb","Type":"ContainerStarted","Data":"fc42a0fae1b73d0fc7e7f9fc5e385528817797f924e639c91fd86732716ecf6e"} Jul 10 02:39:57.121886 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.121866 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerStarted","Data":"97bdb5c840053fd33118b2bd52e0d27ddecfd9f50da69970f8e99f44c60fe003"} Jul 10 02:39:57.123992 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.123973 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wt5wx" event={"ID":"a9cf5549-afa4-46fa-aff4-270bfe266cbe","Type":"ContainerStarted","Data":"0509a890fba798509317e0d211d7be1d0ec154030a85b20b195a91e32e7d52a8"} Jul 10 02:39:57.125109 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.125088 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"9c8712ed006b7fcf35b3f5410d30f14210bd8cfd3d693fba8d2d4835f7c7c21f"} Jul 10 02:39:57.127566 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.127525 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" event={"ID":"73d8f94b-573d-4917-be6c-b3f528d1f167","Type":"ContainerStarted","Data":"bceb52a5edb3fb0436204d0dce8af4cd4ef0b587d97301fac668021a76bd4c04"} Jul 10 02:39:57.128578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.128555 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k6qd5" event={"ID":"fb1de517-e278-4495-bc0e-02c07d5964d3","Type":"ContainerStarted","Data":"345d8d3cc7daa04d97fe3f437d31328a292303e09fa9a0d8944fa82b0e79e538"} Jul 10 02:39:57.130186 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.130147 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-175.ec2.internal" podStartSLOduration=3.130136588 podStartE2EDuration="3.130136588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:57.129742788 +0000 UTC m=+3.838551337" watchObservedRunningTime="2026-07-10 02:39:57.130136588 +0000 UTC m=+3.838945123" Jul 10 02:39:57.391749 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.391071 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:57.391749 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.391225 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:57.391749 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.391290 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:39:59.391271172 +0000 UTC m=+6.100079705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:57.492261 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.492213 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:57.492261 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.492265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:57.492506 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.492461 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:39:57.492506 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.492484 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:39:57.492506 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.492497 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:57.492671 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.492554 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:39:59.492536494 +0000 UTC m=+6.201345017 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:57.493005 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.492965 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:57.493087 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:57.493021 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:39:59.493004965 +0000 UTC m=+6.201813488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:57.791530 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.791377 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:34:55 +0000 UTC" deadline="2027-12-24 14:50:40.436258846 +0000 UTC" Jul 10 02:39:57.791530 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:57.791439 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12780h10m42.644824348s" Jul 10 02:39:58.106066 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:58.105765 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:58.106066 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:58.105902 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:39:58.151120 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:58.151078 2585 generic.go:356] "Generic (PLEG): container finished" podID="ddd074d505fbaa3df61bf060561b2fdc" containerID="5072ef4e86fee58ff57b00c7791e8c7dc61493e38deeafd562251d02b55faf9b" exitCode=0 Jul 10 02:39:58.152400 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:58.152175 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" event={"ID":"ddd074d505fbaa3df61bf060561b2fdc","Type":"ContainerDied","Data":"5072ef4e86fee58ff57b00c7791e8c7dc61493e38deeafd562251d02b55faf9b"} Jul 10 02:39:59.105318 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:59.105286 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:59.105768 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.105419 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:39:59.105837 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:59.105819 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:59.105948 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.105925 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:39:59.180874 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:59.180429 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" event={"ID":"ddd074d505fbaa3df61bf060561b2fdc","Type":"ContainerStarted","Data":"f2ebe8145dd8bb437e2b8639e5189ae0f2740c0f1305fe7a31144ebaf86c18ef"} Jul 10 02:39:59.407773 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:59.407146 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:39:59.407773 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.407288 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:59.407773 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.407348 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:03.407329531 +0000 UTC m=+10.116138058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:39:59.507952 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:59.507909 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:39:59.508134 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:39:59.507971 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:39:59.508134 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.508126 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:39:59.508246 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.508144 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:39:59.508246 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.508159 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:59.508246 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.508215 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:03.508196872 +0000 UTC m=+10.217005390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:39:59.508896 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.508630 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:39:59.508896 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:39:59.508701 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:03.508684583 +0000 UTC m=+10.217493098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:00.115592 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:00.114866 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:00.115592 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:00.115127 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:01.106879 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:01.106379 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:01.106879 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:01.106412 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:01.106879 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:01.106512 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:01.106879 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:01.106640 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:02.105829 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:02.105742 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:02.106251 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:02.105903 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:03.105698 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:03.105666 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:03.105881 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:03.105671 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:03.105881 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.105804 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:03.105881 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.105841 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:03.440397 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:03.439808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:03.440397 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.439955 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:40:03.440397 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.440019 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:11.44000041 +0000 UTC m=+18.148808939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:40:03.540537 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:03.540485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:03.540537 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:03.540540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:03.540985 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.540743 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:03.540985 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.540817 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:11.540796089 +0000 UTC m=+18.249604626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:03.541184 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.540750 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:40:03.541184 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.541156 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:40:03.541184 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.541175 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:40:03.541382 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:03.541265 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:11.541248038 +0000 UTC m=+18.250056566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:40:04.107331 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:04.106868 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:04.107331 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:04.106994 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:05.105459 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:05.105353 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:05.105609 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:05.105467 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:05.105609 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:05.105556 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:05.105748 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:05.105630 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:06.105907 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:06.105872 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:06.106347 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:06.106024 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:07.105809 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:07.105775 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:07.105974 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:07.105778 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:07.105974 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:07.105890 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:07.106373 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:07.105982 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:08.108421 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:08.108394 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:08.108852 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:08.108517 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:09.106174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:09.106131 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:09.106370 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:09.106139 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:09.106370 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:09.106280 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:09.106370 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:09.106343 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:10.105764 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:10.105726 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:10.106243 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:10.105879 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:11.105935 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:11.105905 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:11.106328 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.105997 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:11.106328 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:11.106050 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:11.106328 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.106133 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:11.499438 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:11.499359 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:11.499699 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.499510 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:40:11.499699 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.499569 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:27.499552054 +0000 UTC m=+34.208360582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:40:11.600012 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:11.599974 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:11.600012 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:11.600014 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:11.600250 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.600140 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:11.600250 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.600172 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:40:11.600250 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.600187 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:40:11.600250 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.600212 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:27.600195512 +0000 UTC m=+34.309004031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:11.600250 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.600224 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:40:11.600484 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:11.600266 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:27.600254367 +0000 UTC m=+34.309062881 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:40:12.105579 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:12.105550 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:12.105840 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:12.105699 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:13.106123 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:13.106077 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:13.106123 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:13.106125 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:13.106698 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:13.106217 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:13.106698 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:13.106348 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:14.106452 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:14.106419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:14.106949 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:14.106540 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:15.106216 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:15.106177 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:15.106390 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:15.106290 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:15.106390 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:15.106320 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:15.106520 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:15.106389 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:16.106241 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.105924 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:16.106382 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:16.106357 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:16.215912 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.215880 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-857wz" event={"ID":"c39ba390-03ea-466c-80c1-8dfdb19e32dd","Type":"ContainerStarted","Data":"dce72548ebaffab305801c33e00650a4f954fd2add16e09d604ad1a04c9e51f7"} Jul 10 02:40:16.217540 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.217498 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-69ntc" event={"ID":"41878b22-23b8-43fc-b63d-e753fd7c5b5c","Type":"ContainerStarted","Data":"909e2415e8dd773e517748c0e46be2162ba43c9cdf9f7116b239b16b3eaa8837"} Jul 10 02:40:16.219057 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.219010 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qlcvg" event={"ID":"06dbb287-4849-421f-9e0a-d60266e1b45b","Type":"ContainerStarted","Data":"5607cad741a9639eda52ff26e0ad7de2762cb328da058d62e9a7893033f54212"} Jul 10 02:40:16.220712 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.220686 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" event={"ID":"dfeee125-4ac5-414e-b804-fe0b83d82ffb","Type":"ContainerStarted","Data":"96b068aae1c7434e984cd19bf4364fd0d8b2fad8d3cc13b8132d5b3a3a535326"} Jul 10 02:40:16.222421 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.222396 2585 generic.go:356] "Generic (PLEG): container finished" podID="3689d3fd-6681-4c30-aa1f-53d31bae8186" containerID="32e01738875200dc5386096c66ef738c05ec91b014bf7de1cc8ad5e8ad680b7e" exitCode=0 Jul 10 02:40:16.222541 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.222499 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerDied","Data":"32e01738875200dc5386096c66ef738c05ec91b014bf7de1cc8ad5e8ad680b7e"} Jul 10 02:40:16.226508 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.226326 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"42ad78ec35dbfcc3ca0d7bc8990b5bac372fa7ca937399eb3f5404dbfe5a5565"} Jul 10 02:40:16.226508 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.226390 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"2373bc631b927cd27d39d7db38a345d5e201f7d7f0633119d8004e90c2a25d00"} Jul 10 02:40:16.226508 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.226435 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"0f4a9f1b622e4f0e67f654f82c7ca07aba880d4a1eccae1380a2e09c507623cb"} Jul 10 02:40:16.226508 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.226480 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"e74bcb7fc6679228886005939f5c54ecabe7e55f9caa5ea567160026e2e9f65c"} Jul 10 02:40:16.226746 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.226530 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"e3eadb671416dc37d0db5156b53002a49df7d48966736cccae07fbb3a4b5a48f"} Jul 10 02:40:16.226746 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.226592 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"83533bfc8ea8b0938ce5f3e616230f5ccdefb2e177a8abfdb571f00fa3178fa7"} Jul 10 02:40:16.228357 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.228331 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" event={"ID":"73d8f94b-573d-4917-be6c-b3f528d1f167","Type":"ContainerStarted","Data":"38f1d44b42a3e1d39369135c9bdb5afded09ae89f204c2a72624eee5e851de0d"} Jul 10 02:40:16.230128 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.230100 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k6qd5" event={"ID":"fb1de517-e278-4495-bc0e-02c07d5964d3","Type":"ContainerStarted","Data":"043133913e661df1dc4848545763218ee7f51b514ac82c5f184ce8d34a715147"} Jul 10 02:40:16.233518 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.233483 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-175.ec2.internal" podStartSLOduration=22.233454779 podStartE2EDuration="22.233454779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:59.19620636 +0000 UTC m=+5.905014897" watchObservedRunningTime="2026-07-10 02:40:16.233454779 +0000 UTC m=+22.942263309" Jul 10 02:40:16.233887 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.233854 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-857wz" podStartSLOduration=11.862690665 podStartE2EDuration="22.233840011s" totalImagesPullingTime="10.371149346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:16.233599791 +0000 UTC m=+22.942408310" watchObservedRunningTime="2026-07-10 02:40:16.233840011 +0000 UTC m=+22.942648551" Jul 10 02:40:16.249230 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.249194 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k6qd5" podStartSLOduration=3.783588405 podStartE2EDuration="22.249181557s" totalImagesPullingTime="18.465593152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:16.248791636 +0000 UTC m=+22.957600174" watchObservedRunningTime="2026-07-10 02:40:16.249181557 +0000 UTC m=+22.957990093" Jul 10 02:40:16.264549 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.264507 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7jj5p" podStartSLOduration=3.8045345470000003 podStartE2EDuration="22.264490619s" totalImagesPullingTime="18.459956072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:16.264437356 +0000 UTC m=+22.973245894" watchObservedRunningTime="2026-07-10 02:40:16.264490619 +0000 UTC m=+22.973299151" Jul 10 02:40:16.304926 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.304879 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qlcvg" podStartSLOduration=3.8248016590000002 podStartE2EDuration="22.304864013s" totalImagesPullingTime="18.480062354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:16.304496099 +0000 UTC m=+23.013304635" watchObservedRunningTime="2026-07-10 02:40:16.304864013 +0000 UTC m=+23.013672548" Jul 10 02:40:16.318112 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.318065 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-69ntc" podStartSLOduration=3.86226973 podStartE2EDuration="22.318049674s" totalImagesPullingTime="18.455779944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:16.318029485 +0000 UTC m=+23.026838018" watchObservedRunningTime="2026-07-10 02:40:16.318049674 +0000 UTC m=+23.026858206" Jul 10 02:40:16.764398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.764366 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-857wz" Jul 10 02:40:16.765085 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.765020 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-857wz" Jul 10 02:40:16.894326 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.894296 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 02:40:16.974047 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.973866 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T02:40:16.894318927Z","UUID":"63422ca9-5412-4719-8abf-1f144d0f554f","Handler":null,"Name":"","Endpoint":""} Jul 10 02:40:16.975733 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.975711 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 02:40:16.975859 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:16.975746 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 02:40:17.105634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.105555 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:17.105634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.105583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:17.105869 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:17.105701 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:17.105869 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:17.105824 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:17.234152 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.234109 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wt5wx" event={"ID":"a9cf5549-afa4-46fa-aff4-270bfe266cbe","Type":"ContainerStarted","Data":"34d328b87b831a3345424d838013b2de43cb40f6d1db601b2274dc14d3efcbb6"} Jul 10 02:40:17.235921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.235891 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" event={"ID":"dfeee125-4ac5-414e-b804-fe0b83d82ffb","Type":"ContainerStarted","Data":"f8e83c5c5ea3e13ef04303866a186391b928254b85260b393db9fcdff7f2b663"} Jul 10 02:40:17.236506 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.236488 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-857wz" Jul 10 02:40:17.236849 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.236834 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-857wz" Jul 10 02:40:17.249264 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:17.249227 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wt5wx" podStartSLOduration=4.7863253 podStartE2EDuration="23.249213528s" totalImagesPullingTime="18.462888228s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:17.24886208 +0000 UTC m=+23.957670627" watchObservedRunningTime="2026-07-10 02:40:17.249213528 +0000 UTC m=+23.958022064" Jul 10 02:40:18.105610 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:18.105581 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:18.105793 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:18.105739 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:18.240672 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:18.240166 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" event={"ID":"dfeee125-4ac5-414e-b804-fe0b83d82ffb","Type":"ContainerStarted","Data":"09a41878d9e6a4d06b051dcb139c5fb5465c850a53f5afa39c56c611cf491651"} Jul 10 02:40:18.259935 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:18.259877 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlwvq" podStartSLOduration=3.684174666 podStartE2EDuration="24.25985561s" totalImagesPullingTime="20.575680944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:18.259355899 +0000 UTC m=+24.968164449" watchObservedRunningTime="2026-07-10 02:40:18.25985561 +0000 UTC m=+24.968664147" Jul 10 02:40:19.105971 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:19.105937 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:19.106158 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:19.105958 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:19.106158 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:19.106081 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:19.106158 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:19.106109 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:19.246447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:19.246410 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"13763ce33425cc63c560c9bbcd3aa4634ca616a1b5f4bb5b9f60805923642667"} Jul 10 02:40:20.105767 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:20.105506 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:20.105947 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:20.105902 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:21.105737 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.105521 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:21.106555 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.105521 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:21.106555 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:21.105835 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:21.106555 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:21.105925 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:21.251982 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.251948 2585 generic.go:356] "Generic (PLEG): container finished" podID="3689d3fd-6681-4c30-aa1f-53d31bae8186" containerID="182e65c784cbd2798253f1b66044a271e53c07159ba76e2675cae24a7b24504e" exitCode=0 Jul 10 02:40:21.252166 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.252037 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerDied","Data":"182e65c784cbd2798253f1b66044a271e53c07159ba76e2675cae24a7b24504e"} Jul 10 02:40:21.255745 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.255719 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" event={"ID":"9cbc0665-a310-4e2c-97f8-acb6fcdfaae7","Type":"ContainerStarted","Data":"f3736b1eec116abbcfb54c43b4bb42d0b62ec0ace2deec59c81343ceb6d9a2c7"} Jul 10 02:40:21.256000 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.255977 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:40:21.256064 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.256040 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:40:21.256121 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.256078 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:40:21.271257 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.271232 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:40:21.271392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.271329 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:40:21.298053 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:21.298003 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" podStartSLOduration=8.696012761 podStartE2EDuration="27.297987634s" totalImagesPullingTime="18.601974873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:21.297419221 +0000 UTC m=+28.006227756" watchObservedRunningTime="2026-07-10 02:40:21.297987634 +0000 UTC m=+28.006796170" Jul 10 02:40:22.106300 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.106267 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:22.106692 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:22.106430 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:22.470909 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.470878 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wd6fm"] Jul 10 02:40:22.471086 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.470989 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:22.471128 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:22.471090 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:22.473773 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.473739 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5wkwz"] Jul 10 02:40:22.473911 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.473854 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:22.473976 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:22.473953 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:22.474485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.474461 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zn6fq"] Jul 10 02:40:22.474559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:22.474549 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:22.474687 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:22.474666 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:23.262045 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:23.262012 2585 generic.go:356] "Generic (PLEG): container finished" podID="3689d3fd-6681-4c30-aa1f-53d31bae8186" containerID="1e9efde05306517e0a12316e42572f2c556e0f3c95a96ef6b50344821832172a" exitCode=0 Jul 10 02:40:23.262478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:23.262094 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerDied","Data":"1e9efde05306517e0a12316e42572f2c556e0f3c95a96ef6b50344821832172a"} Jul 10 02:40:24.106958 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:24.106918 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:24.107154 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:24.106987 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:24.107154 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:24.106998 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:24.107154 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:24.107106 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:24.107154 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:24.107140 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:24.107364 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:24.107190 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:25.268364 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:25.268186 2585 generic.go:356] "Generic (PLEG): container finished" podID="3689d3fd-6681-4c30-aa1f-53d31bae8186" containerID="5dba706a6a49fc64496b9ef399913be4a5ca14c4a568fc78df9f3f5c8f9b144c" exitCode=0 Jul 10 02:40:25.268767 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:25.268268 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerDied","Data":"5dba706a6a49fc64496b9ef399913be4a5ca14c4a568fc78df9f3f5c8f9b144c"} Jul 10 02:40:26.105520 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:26.105485 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:26.105728 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:26.105486 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:26.105728 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:26.105638 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zn6fq" podUID="f53b3aa6-f527-411f-ad5d-0f4bd076dd0b" Jul 10 02:40:26.105728 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:26.105485 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:26.105902 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:26.105742 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5wkwz" podUID="1eddeeef-12e8-4c5f-b692-5f52c8642bbe" Jul 10 02:40:26.105902 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:26.105829 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wd6fm" podUID="0991902f-cbfb-41d6-9399-519aac9b646b" Jul 10 02:40:27.516111 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:27.516068 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:27.516743 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.516173 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:40:27.516743 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.516260 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs podName:f53b3aa6-f527-411f-ad5d-0f4bd076dd0b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:59.516234976 +0000 UTC m=+66.225043493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs") pod "network-metrics-daemon-zn6fq" (UID: "f53b3aa6-f527-411f-ad5d-0f4bd076dd0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 02:40:27.616972 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:27.616934 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:27.617171 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:27.616983 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:27.617171 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.617077 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:27.617171 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.617120 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 02:40:27.617171 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.617138 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 02:40:27.617171 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.617151 2585 projected.go:196] Error preparing data for projected volume kube-api-access-mxjr8 for pod openshift-network-diagnostics/network-check-target-5wkwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:40:27.617371 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.617152 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret podName:0991902f-cbfb-41d6-9399-519aac9b646b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:59.61713188 +0000 UTC m=+66.325940408 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret") pod "global-pull-secret-syncer-wd6fm" (UID: "0991902f-cbfb-41d6-9399-519aac9b646b") : object "kube-system"/"original-pull-secret" not registered Jul 10 02:40:27.617371 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:27.617194 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8 podName:1eddeeef-12e8-4c5f-b692-5f52c8642bbe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:59.617184854 +0000 UTC m=+66.325993371 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mxjr8" (UniqueName: "kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8") pod "network-check-target-5wkwz" (UID: "1eddeeef-12e8-4c5f-b692-5f52c8642bbe") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 02:40:28.074514 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.074483 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-175.ec2.internal" event="NodeReady" Jul 10 02:40:28.074697 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.074644 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 02:40:28.106359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.106328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:28.106533 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.106328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:28.106533 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.106328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:28.109311 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.109286 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 02:40:28.109311 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.109311 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 02:40:28.109515 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.109370 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 02:40:28.109770 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.109620 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 02:40:28.109770 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.109682 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jw76v\"" Jul 10 02:40:28.109923 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.109790 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9pgv\"" Jul 10 02:40:28.117080 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.117052 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4w9w5"] Jul 10 02:40:28.119862 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.119843 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jktcm"] Jul 10 02:40:28.119985 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.119968 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.121804 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.121782 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.122918 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.122883 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 02:40:28.123179 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.123058 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8l52l\"" Jul 10 02:40:28.123179 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.123094 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 02:40:28.124419 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.124401 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2tgb9\"" Jul 10 02:40:28.124509 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.124421 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 02:40:28.125429 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.125410 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 02:40:28.125630 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.125611 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 02:40:28.131841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.131822 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4w9w5"] Jul 10 02:40:28.134502 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.134468 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jktcm"] Jul 10 02:40:28.221234 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.221107 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33303bd9-1b00-4884-a4bb-028e5620fafe-config-volume\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.221395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.221244 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd-cert\") pod \"ingress-canary-jktcm\" (UID: \"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd\") " pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.221395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.221277 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8fcm\" (UniqueName: \"kubernetes.io/projected/f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd-kube-api-access-l8fcm\") pod \"ingress-canary-jktcm\" (UID: \"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd\") " pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.221395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.221297 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.221395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.221314 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33303bd9-1b00-4884-a4bb-028e5620fafe-tmp-dir\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.221546 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.221417 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gzq4\" (UniqueName: \"kubernetes.io/projected/33303bd9-1b00-4884-a4bb-028e5620fafe-kube-api-access-4gzq4\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.322316 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.322272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gzq4\" (UniqueName: \"kubernetes.io/projected/33303bd9-1b00-4884-a4bb-028e5620fafe-kube-api-access-4gzq4\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.322516 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.322337 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33303bd9-1b00-4884-a4bb-028e5620fafe-config-volume\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.322516 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.322379 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd-cert\") pod \"ingress-canary-jktcm\" (UID: \"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd\") " pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.322516 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.322416 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8fcm\" (UniqueName: \"kubernetes.io/projected/f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd-kube-api-access-l8fcm\") pod \"ingress-canary-jktcm\" (UID: \"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd\") " pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.322779 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.322738 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.322909 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.322787 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33303bd9-1b00-4884-a4bb-028e5620fafe-tmp-dir\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.323047 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.323023 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33303bd9-1b00-4884-a4bb-028e5620fafe-config-volume\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.323147 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.323076 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33303bd9-1b00-4884-a4bb-028e5620fafe-tmp-dir\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.323199 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:28.323178 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:40:28.323267 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:28.323259 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls podName:33303bd9-1b00-4884-a4bb-028e5620fafe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:28.823238444 +0000 UTC m=+35.532046975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls") pod "dns-default-4w9w5" (UID: "33303bd9-1b00-4884-a4bb-028e5620fafe") : secret "dns-default-metrics-tls" not found Jul 10 02:40:28.327450 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.327427 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd-cert\") pod \"ingress-canary-jktcm\" (UID: \"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd\") " pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.331035 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.331009 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gzq4\" (UniqueName: \"kubernetes.io/projected/33303bd9-1b00-4884-a4bb-028e5620fafe-kube-api-access-4gzq4\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.332321 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.332296 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8fcm\" (UniqueName: \"kubernetes.io/projected/f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd-kube-api-access-l8fcm\") pod \"ingress-canary-jktcm\" (UID: \"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd\") " pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.448830 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.448791 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jktcm" Jul 10 02:40:28.826032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:28.825987 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:28.826810 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:28.826169 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:40:28.826810 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:28.826253 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls podName:33303bd9-1b00-4884-a4bb-028e5620fafe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:29.826229237 +0000 UTC m=+36.535037754 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls") pod "dns-default-4w9w5" (UID: "33303bd9-1b00-4884-a4bb-028e5620fafe") : secret "dns-default-metrics-tls" not found Jul 10 02:40:29.308569 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:29.308422 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jktcm"] Jul 10 02:40:29.311258 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:29.311234 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28708fe_6f76_4cc0_b5fe_a9e15c62ecdd.slice/crio-07b8aff8475af545a737c7fea32420cc5a31314e76290768d88d5faac496d110 WatchSource:0}: Error finding container 07b8aff8475af545a737c7fea32420cc5a31314e76290768d88d5faac496d110: Status 404 returned error can't find the container with id 07b8aff8475af545a737c7fea32420cc5a31314e76290768d88d5faac496d110 Jul 10 02:40:29.833523 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:29.833488 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:29.834016 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:29.833603 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:40:29.834016 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:29.833679 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls podName:33303bd9-1b00-4884-a4bb-028e5620fafe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:31.833643499 +0000 UTC m=+38.542452016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls") pod "dns-default-4w9w5" (UID: "33303bd9-1b00-4884-a4bb-028e5620fafe") : secret "dns-default-metrics-tls" not found Jul 10 02:40:29.930710 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:29.930666 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:30.016021 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:30.015961 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:30.116984 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:30.116889 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:30.282746 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:30.282713 2585 generic.go:356] "Generic (PLEG): container finished" podID="3689d3fd-6681-4c30-aa1f-53d31bae8186" containerID="cf4e4d936ddad90050c02a91c0655fe7f775af0ab113cd1ee745f9b56883ec47" exitCode=0 Jul 10 02:40:30.282909 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:30.282801 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerDied","Data":"cf4e4d936ddad90050c02a91c0655fe7f775af0ab113cd1ee745f9b56883ec47"} Jul 10 02:40:30.284685 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:30.284642 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jktcm" event={"ID":"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd","Type":"ContainerStarted","Data":"07b8aff8475af545a737c7fea32420cc5a31314e76290768d88d5faac496d110"} Jul 10 02:40:31.261715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:31.261677 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:31.290566 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:31.290535 2585 generic.go:356] "Generic (PLEG): container finished" podID="3689d3fd-6681-4c30-aa1f-53d31bae8186" containerID="44a475ec21e0d74f51ad0b68e68a8320be156d379799f68eabe99db9d53f494e" exitCode=0 Jul 10 02:40:31.290743 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:31.290605 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerDied","Data":"44a475ec21e0d74f51ad0b68e68a8320be156d379799f68eabe99db9d53f494e"} Jul 10 02:40:31.336693 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:31.336630 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:31.849448 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:31.849215 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:31.849620 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:31.849373 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:40:31.849620 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:31.849545 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls podName:33303bd9-1b00-4884-a4bb-028e5620fafe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:35.849521765 +0000 UTC m=+42.558330286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls") pod "dns-default-4w9w5" (UID: "33303bd9-1b00-4884-a4bb-028e5620fafe") : secret "dns-default-metrics-tls" not found Jul 10 02:40:32.163489 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.163402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:32.219371 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.219334 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:32.294738 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.294708 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jktcm" event={"ID":"f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd","Type":"ContainerStarted","Data":"6bd3b20127ee66d6f0c0df8ee80db5e9bd9f96a0ccfd3c43ed55a4d1c225b8db"} Jul 10 02:40:32.298237 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.298210 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" event={"ID":"3689d3fd-6681-4c30-aa1f-53d31bae8186","Type":"ContainerStarted","Data":"d3d8f9bde2fd1d57fc86d217097713f1623faab2f1c797bd891b9313071849ba"} Jul 10 02:40:32.310131 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.310083 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jktcm" podStartSLOduration=1.940831771 podStartE2EDuration="4.31006851s" totalImagesPullingTime="2.369236739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:32.309599905 +0000 UTC m=+39.018408444" watchObservedRunningTime="2026-07-10 02:40:32.31006851 +0000 UTC m=+39.018877039" Jul 10 02:40:32.331986 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.331935 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2kxtv" podStartSLOduration=9.515397867 podStartE2EDuration="38.331920575s" totalImagesPullingTime="28.816522708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:32.330777435 +0000 UTC m=+39.039585967" watchObservedRunningTime="2026-07-10 02:40:32.331920575 +0000 UTC m=+39.040729105" Jul 10 02:40:32.454205 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.454162 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:32.514731 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:32.514700 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:33.039494 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.039454 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8"] Jul 10 02:40:33.052281 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.052245 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8"] Jul 10 02:40:33.052456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.052352 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" Jul 10 02:40:33.055146 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.055115 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:40:33.056299 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.056275 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 02:40:33.056418 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.056343 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hmt55\"" Jul 10 02:40:33.142293 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.142252 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-xtwmx"] Jul 10 02:40:33.158244 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.158210 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-547685df6b-b8cpb"] Jul 10 02:40:33.158384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.158328 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.159087 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.159061 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j97lb\" (UniqueName: \"kubernetes.io/projected/2bda1c2d-6412-4f63-8f99-10cbf44fa771-kube-api-access-j97lb\") pod \"volume-data-source-validator-78495597f6-76lb8\" (UID: \"2bda1c2d-6412-4f63-8f99-10cbf44fa771\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" Jul 10 02:40:33.161431 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.161409 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fq767\"" Jul 10 02:40:33.161550 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.161448 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 02:40:33.161594 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.161570 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 02:40:33.161711 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.161696 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 02:40:33.168227 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.168206 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:40:33.171212 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.171191 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-xtwmx"] Jul 10 02:40:33.171319 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.171264 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-547685df6b-b8cpb"] Jul 10 02:40:33.171378 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.171356 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.171883 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.171861 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 02:40:33.174458 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.174440 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w6ngk\"" Jul 10 02:40:33.174619 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.174607 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 02:40:33.174769 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.174754 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 02:40:33.174944 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.174930 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 02:40:33.181256 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.181240 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 02:40:33.238065 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.238042 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf"] Jul 10 02:40:33.256186 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.256161 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh"] Jul 10 02:40:33.256302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.256284 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" Jul 10 02:40:33.259332 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259315 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jgm68\"" Jul 10 02:40:33.259465 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259442 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-certificates\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.259524 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259486 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-image-registry-private-configuration\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.259569 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259556 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e54d5b6-b253-4343-a8b0-8c22d69bec95-serving-cert\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.259637 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259620 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d542f2a5-c5fe-4e93-b2e7-3209085724c3-ca-trust-extracted\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.259707 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259659 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-bound-sa-token\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.259707 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259694 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j97lb\" (UniqueName: \"kubernetes.io/projected/2bda1c2d-6412-4f63-8f99-10cbf44fa771-kube-api-access-j97lb\") pod \"volume-data-source-validator-78495597f6-76lb8\" (UID: \"2bda1c2d-6412-4f63-8f99-10cbf44fa771\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" Jul 10 02:40:33.259774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259733 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e54d5b6-b253-4343-a8b0-8c22d69bec95-config\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.259774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259757 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e54d5b6-b253-4343-a8b0-8c22d69bec95-trusted-ca\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.259869 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259793 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-566gv\" (UniqueName: \"kubernetes.io/projected/8e54d5b6-b253-4343-a8b0-8c22d69bec95-kube-api-access-566gv\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.259922 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259876 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-installation-pull-secrets\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.259973 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259925 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.260020 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.259977 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-trusted-ca\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.260073 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.260015 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2l79\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-kube-api-access-d2l79\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.275769 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.275749 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j97lb\" (UniqueName: \"kubernetes.io/projected/2bda1c2d-6412-4f63-8f99-10cbf44fa771-kube-api-access-j97lb\") pod \"volume-data-source-validator-78495597f6-76lb8\" (UID: \"2bda1c2d-6412-4f63-8f99-10cbf44fa771\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" Jul 10 02:40:33.278378 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.278357 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d"] Jul 10 02:40:33.278505 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.278483 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.281296 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.281277 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 02:40:33.281545 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.281527 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 02:40:33.281627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.281534 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qw5wl\"" Jul 10 02:40:33.281627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.281560 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 02:40:33.281627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.281565 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:40:33.304355 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.304292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf"] Jul 10 02:40:33.304355 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.304324 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d"] Jul 10 02:40:33.304355 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.304343 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh"] Jul 10 02:40:33.305034 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.304430 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.307019 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.307003 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 02:40:33.307290 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.307275 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 02:40:33.307290 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.307280 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:40:33.307398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.307277 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 02:40:33.307398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.307320 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6s8m7\"" Jul 10 02:40:33.360758 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360732 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r5rk\" (UniqueName: \"kubernetes.io/projected/b1972ee0-9c38-4c7d-be80-4a6cebda905f-kube-api-access-9r5rk\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.360896 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e54d5b6-b253-4343-a8b0-8c22d69bec95-serving-cert\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.360896 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360806 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d542f2a5-c5fe-4e93-b2e7-3209085724c3-ca-trust-extracted\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.360896 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360846 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-bound-sa-token\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.360896 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e54d5b6-b253-4343-a8b0-8c22d69bec95-config\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.361101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360899 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e54d5b6-b253-4343-a8b0-8c22d69bec95-trusted-ca\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.361101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-566gv\" (UniqueName: \"kubernetes.io/projected/8e54d5b6-b253-4343-a8b0-8c22d69bec95-kube-api-access-566gv\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.361101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/737f8d61-c6e6-4780-95dd-49507bf10d66-config\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.361101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.360984 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/737f8d61-c6e6-4780-95dd-49507bf10d66-serving-cert\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.361101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361064 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-installation-pull-secrets\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.361317 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361113 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42fbp\" (UniqueName: \"kubernetes.io/projected/737f8d61-c6e6-4780-95dd-49507bf10d66-kube-api-access-42fbp\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.361317 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361174 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d542f2a5-c5fe-4e93-b2e7-3209085724c3-ca-trust-extracted\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.361317 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.361447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361340 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-trusted-ca\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.361447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361378 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2l79\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-kube-api-access-d2l79\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.361447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361404 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1972ee0-9c38-4c7d-be80-4a6cebda905f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.361447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361444 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-certificates\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.361625 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1972ee0-9c38-4c7d-be80-4a6cebda905f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.361706 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361672 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8e54d5b6-b253-4343-a8b0-8c22d69bec95-config\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.361758 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361732 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" Jul 10 02:40:33.361758 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:33.361480 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:40:33.361848 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:33.361772 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-547685df6b-b8cpb: secret "image-registry-tls" not found Jul 10 02:40:33.361848 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:33.361837 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls podName:d542f2a5-c5fe-4e93-b2e7-3209085724c3 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:33.861818978 +0000 UTC m=+40.570627512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls") pod "image-registry-547685df6b-b8cpb" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3") : secret "image-registry-tls" not found Jul 10 02:40:33.362005 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.361984 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-image-registry-private-configuration\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.362067 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.362031 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drp4q\" (UniqueName: \"kubernetes.io/projected/f985b9a9-3d71-4d13-ab18-3859217e6eff-kube-api-access-drp4q\") pod \"network-check-source-869dcfcdcd-8z5qf\" (UID: \"f985b9a9-3d71-4d13-ab18-3859217e6eff\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" Jul 10 02:40:33.362121 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.362081 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8e54d5b6-b253-4343-a8b0-8c22d69bec95-trusted-ca\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.362297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.362274 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-certificates\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.362435 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.362418 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-trusted-ca\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.363732 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.363706 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8e54d5b6-b253-4343-a8b0-8c22d69bec95-serving-cert\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.363917 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.363900 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-installation-pull-secrets\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.364128 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.364110 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-image-registry-private-configuration\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.372774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.372757 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-bound-sa-token\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.375852 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.375830 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-566gv\" (UniqueName: \"kubernetes.io/projected/8e54d5b6-b253-4343-a8b0-8c22d69bec95-kube-api-access-566gv\") pod \"console-operator-7866497494-xtwmx\" (UID: \"8e54d5b6-b253-4343-a8b0-8c22d69bec95\") " pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.376239 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.376219 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2l79\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-kube-api-access-d2l79\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.393155 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.393124 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:33.463100 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463063 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1972ee0-9c38-4c7d-be80-4a6cebda905f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.463235 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463127 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1972ee0-9c38-4c7d-be80-4a6cebda905f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.463235 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463159 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drp4q\" (UniqueName: \"kubernetes.io/projected/f985b9a9-3d71-4d13-ab18-3859217e6eff-kube-api-access-drp4q\") pod \"network-check-source-869dcfcdcd-8z5qf\" (UID: \"f985b9a9-3d71-4d13-ab18-3859217e6eff\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" Jul 10 02:40:33.463235 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463199 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r5rk\" (UniqueName: \"kubernetes.io/projected/b1972ee0-9c38-4c7d-be80-4a6cebda905f-kube-api-access-9r5rk\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.463399 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463256 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/737f8d61-c6e6-4780-95dd-49507bf10d66-config\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.463399 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463288 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/737f8d61-c6e6-4780-95dd-49507bf10d66-serving-cert\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.463399 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.463323 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42fbp\" (UniqueName: \"kubernetes.io/projected/737f8d61-c6e6-4780-95dd-49507bf10d66-kube-api-access-42fbp\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.464948 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.464693 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1972ee0-9c38-4c7d-be80-4a6cebda905f-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.469379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.468924 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/737f8d61-c6e6-4780-95dd-49507bf10d66-serving-cert\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.469379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.469085 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:33.470758 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.470546 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1972ee0-9c38-4c7d-be80-4a6cebda905f-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.472398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.472373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42fbp\" (UniqueName: \"kubernetes.io/projected/737f8d61-c6e6-4780-95dd-49507bf10d66-kube-api-access-42fbp\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.473441 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.473414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r5rk\" (UniqueName: \"kubernetes.io/projected/b1972ee0-9c38-4c7d-be80-4a6cebda905f-kube-api-access-9r5rk\") pod \"kube-storage-version-migrator-operator-799cb7f958-rp59d\" (UID: \"b1972ee0-9c38-4c7d-be80-4a6cebda905f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.474906 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.474875 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drp4q\" (UniqueName: \"kubernetes.io/projected/f985b9a9-3d71-4d13-ab18-3859217e6eff-kube-api-access-drp4q\") pod \"network-check-source-869dcfcdcd-8z5qf\" (UID: \"f985b9a9-3d71-4d13-ab18-3859217e6eff\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" Jul 10 02:40:33.481983 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.481934 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/737f8d61-c6e6-4780-95dd-49507bf10d66-config\") pod \"service-ca-operator-7695fc6c8c-ws8vh\" (UID: \"737f8d61-c6e6-4780-95dd-49507bf10d66\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.493764 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.493578 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:33.516092 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.516072 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8"] Jul 10 02:40:33.518292 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:33.518265 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bda1c2d_6412_4f63_8f99_10cbf44fa771.slice/crio-332593d292ce6526a7c428a24861086c6e7026e8e6e90623b6d8d4d93e717c8e WatchSource:0}: Error finding container 332593d292ce6526a7c428a24861086c6e7026e8e6e90623b6d8d4d93e717c8e: Status 404 returned error can't find the container with id 332593d292ce6526a7c428a24861086c6e7026e8e6e90623b6d8d4d93e717c8e Jul 10 02:40:33.567208 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.567118 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" Jul 10 02:40:33.587805 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.587769 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" Jul 10 02:40:33.594339 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.594309 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:33.612133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.612095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" Jul 10 02:40:33.632864 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.632784 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-xtwmx"] Jul 10 02:40:33.636406 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:33.636371 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e54d5b6_b253_4343_a8b0_8c22d69bec95.slice/crio-3fe2622f770774ef84ab22b8342647b1f89fc9568d2660f9e73a82ad7298e35a WatchSource:0}: Error finding container 3fe2622f770774ef84ab22b8342647b1f89fc9568d2660f9e73a82ad7298e35a: Status 404 returned error can't find the container with id 3fe2622f770774ef84ab22b8342647b1f89fc9568d2660f9e73a82ad7298e35a Jul 10 02:40:33.719625 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.719592 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf"] Jul 10 02:40:33.724506 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:33.724481 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf985b9a9_3d71_4d13_ab18_3859217e6eff.slice/crio-c2dac393de0236c0a51ab239c58240d2e5ff9ad26e4d9a062f01adbbc0febad8 WatchSource:0}: Error finding container c2dac393de0236c0a51ab239c58240d2e5ff9ad26e4d9a062f01adbbc0febad8: Status 404 returned error can't find the container with id c2dac393de0236c0a51ab239c58240d2e5ff9ad26e4d9a062f01adbbc0febad8 Jul 10 02:40:33.753628 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.753599 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh"] Jul 10 02:40:33.756797 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:33.756770 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737f8d61_c6e6_4780_95dd_49507bf10d66.slice/crio-01717648475f6df9946bbe0d375185e62cf1e35dc1dcec14eaf7a494fdec4dde WatchSource:0}: Error finding container 01717648475f6df9946bbe0d375185e62cf1e35dc1dcec14eaf7a494fdec4dde: Status 404 returned error can't find the container with id 01717648475f6df9946bbe0d375185e62cf1e35dc1dcec14eaf7a494fdec4dde Jul 10 02:40:33.780733 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.780708 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d"] Jul 10 02:40:33.783823 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:33.783796 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1972ee0_9c38_4c7d_be80_4a6cebda905f.slice/crio-46ca9b327ae4b9ffd700eb43dbf18c7db1ccb9af49044fc9c3901df7e82e448a WatchSource:0}: Error finding container 46ca9b327ae4b9ffd700eb43dbf18c7db1ccb9af49044fc9c3901df7e82e448a: Status 404 returned error can't find the container with id 46ca9b327ae4b9ffd700eb43dbf18c7db1ccb9af49044fc9c3901df7e82e448a Jul 10 02:40:33.867341 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:33.867260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:33.867470 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:33.867408 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:40:33.867470 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:33.867427 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-547685df6b-b8cpb: secret "image-registry-tls" not found Jul 10 02:40:33.867547 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:33.867483 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls podName:d542f2a5-c5fe-4e93-b2e7-3209085724c3 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:34.867468794 +0000 UTC m=+41.576277308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls") pod "image-registry-547685df6b-b8cpb" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3") : secret "image-registry-tls" not found Jul 10 02:40:34.304605 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:34.304567 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" event={"ID":"737f8d61-c6e6-4780-95dd-49507bf10d66","Type":"ContainerStarted","Data":"01717648475f6df9946bbe0d375185e62cf1e35dc1dcec14eaf7a494fdec4dde"} Jul 10 02:40:34.305790 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:34.305763 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" event={"ID":"f985b9a9-3d71-4d13-ab18-3859217e6eff","Type":"ContainerStarted","Data":"c2dac393de0236c0a51ab239c58240d2e5ff9ad26e4d9a062f01adbbc0febad8"} Jul 10 02:40:34.306925 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:34.306899 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" event={"ID":"b1972ee0-9c38-4c7d-be80-4a6cebda905f","Type":"ContainerStarted","Data":"46ca9b327ae4b9ffd700eb43dbf18c7db1ccb9af49044fc9c3901df7e82e448a"} Jul 10 02:40:34.308039 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:34.308020 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xtwmx" event={"ID":"8e54d5b6-b253-4343-a8b0-8c22d69bec95","Type":"ContainerStarted","Data":"3fe2622f770774ef84ab22b8342647b1f89fc9568d2660f9e73a82ad7298e35a"} Jul 10 02:40:34.309197 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:34.309176 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" event={"ID":"2bda1c2d-6412-4f63-8f99-10cbf44fa771","Type":"ContainerStarted","Data":"332593d292ce6526a7c428a24861086c6e7026e8e6e90623b6d8d4d93e717c8e"} Jul 10 02:40:34.876917 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:34.876809 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:34.877184 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:34.876994 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:40:34.877184 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:34.877019 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-547685df6b-b8cpb: secret "image-registry-tls" not found Jul 10 02:40:34.877184 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:34.877086 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls podName:d542f2a5-c5fe-4e93-b2e7-3209085724c3 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:36.877063967 +0000 UTC m=+43.585872484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls") pod "image-registry-547685df6b-b8cpb" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3") : secret "image-registry-tls" not found Jul 10 02:40:35.002487 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:35.002450 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:35.060901 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:35.060863 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:35.887395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:35.887250 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:35.887932 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:35.887408 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:40:35.887932 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:35.887488 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls podName:33303bd9-1b00-4884-a4bb-028e5620fafe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:43.887467007 +0000 UTC m=+50.596275527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls") pod "dns-default-4w9w5" (UID: "33303bd9-1b00-4884-a4bb-028e5620fafe") : secret "dns-default-metrics-tls" not found Jul 10 02:40:36.896056 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:36.895974 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:36.896428 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:36.896146 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:40:36.896428 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:36.896168 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-547685df6b-b8cpb: secret "image-registry-tls" not found Jul 10 02:40:36.896428 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:36.896232 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls podName:d542f2a5-c5fe-4e93-b2e7-3209085724c3 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:40.89621449 +0000 UTC m=+47.605023004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls") pod "image-registry-547685df6b-b8cpb" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3") : secret "image-registry-tls" not found Jul 10 02:40:37.687006 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.686971 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7vr8g"] Jul 10 02:40:37.707133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.707102 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7vr8g"] Jul 10 02:40:37.707299 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.707240 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.709830 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.709789 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 02:40:37.709983 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.709880 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c8l6g\"" Jul 10 02:40:37.710040 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.709982 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 02:40:37.710892 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.710872 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 02:40:37.711011 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.710907 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 02:40:37.724209 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.724186 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:37.778925 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.778890 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:37.804704 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.804672 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.804885 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.804801 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdrsb\" (UniqueName: \"kubernetes.io/projected/05fdce4d-a91e-46e1-9326-0c151298684b-kube-api-access-kdrsb\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.804885 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.804833 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05fdce4d-a91e-46e1-9326-0c151298684b-data-volume\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.805056 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.804917 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.805056 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.804960 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05fdce4d-a91e-46e1-9326-0c151298684b-crio-socket\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.905844 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.905804 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.905869 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05fdce4d-a91e-46e1-9326-0c151298684b-crio-socket\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.905915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:37.905955 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:37.906031 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:37.906043 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:38.406021183 +0000 UTC m=+45.114829720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : configmap "kube-rbac-proxy" not found Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.906064 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05fdce4d-a91e-46e1-9326-0c151298684b-crio-socket\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:37.906071 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:38.406060174 +0000 UTC m=+45.114868691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : secret "insights-runtime-extractor-tls" not found Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.906162 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdrsb\" (UniqueName: \"kubernetes.io/projected/05fdce4d-a91e-46e1-9326-0c151298684b-kube-api-access-kdrsb\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.906297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.906201 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05fdce4d-a91e-46e1-9326-0c151298684b-data-volume\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.906688 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.906482 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05fdce4d-a91e-46e1-9326-0c151298684b-data-volume\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:37.915527 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:37.915499 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdrsb\" (UniqueName: \"kubernetes.io/projected/05fdce4d-a91e-46e1-9326-0c151298684b-kube-api-access-kdrsb\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:38.095454 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:38.095366 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:38.410868 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:38.410785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:38.410868 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:38.410841 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:38.411102 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:38.410932 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:40:38.411102 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:38.411009 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:39.41098666 +0000 UTC m=+46.119795194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : configmap "kube-rbac-proxy" not found Jul 10 02:40:38.411102 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:38.411020 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:40:38.411102 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:38.411077 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:39.411059818 +0000 UTC m=+46.119868354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : secret "insights-runtime-extractor-tls" not found Jul 10 02:40:38.934841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:38.934806 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:38.988067 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:38.988032 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:39.332293 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.332261 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" event={"ID":"2bda1c2d-6412-4f63-8f99-10cbf44fa771","Type":"ContainerStarted","Data":"9bcbbf391f801bb413fcd96f23fc8df7a0d0f4a80c5073a031a70a825550ee94"} Jul 10 02:40:39.355359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.355314 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-76lb8" podStartSLOduration=4.377577275 podStartE2EDuration="6.355300251s" totalImagesPullingTime="1.977722976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:39.354935 +0000 UTC m=+46.063743539" watchObservedRunningTime="2026-07-10 02:40:39.355300251 +0000 UTC m=+46.064108787" Jul 10 02:40:39.419035 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.418991 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:39.419196 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.419080 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:39.419257 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:39.419193 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:40:39.419257 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:39.419248 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:41.41922876 +0000 UTC m=+48.128037275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : secret "insights-runtime-extractor-tls" not found Jul 10 02:40:39.419373 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:39.419298 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 02:40:39.419373 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:39.419327 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:41.419318352 +0000 UTC m=+48.128126866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : configmap "kube-rbac-proxy" not found Jul 10 02:40:39.454393 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.454300 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:39.470641 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.470616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k6qd5_fb1de517-e278-4495-bc0e-02c07d5964d3/dns-node-resolver/0.log" Jul 10 02:40:39.514822 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:39.514775 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:40.070418 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.070390 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-69ntc_41878b22-23b8-43fc-b63d-e753fd7c5b5c/node-ca/0.log" Jul 10 02:40:40.337603 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.337510 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" event={"ID":"737f8d61-c6e6-4780-95dd-49507bf10d66","Type":"ContainerStarted","Data":"6f68af5820389127916cc11f9fabba5667e245ee79a36c0f8276f187a1829adf"} Jul 10 02:40:40.339205 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.339174 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" event={"ID":"f985b9a9-3d71-4d13-ab18-3859217e6eff","Type":"ContainerStarted","Data":"399e89189c0d6f559689c2a9c8c083d6d51fbb5af90111a3cb98f214633928a3"} Jul 10 02:40:40.340776 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.340752 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" event={"ID":"b1972ee0-9c38-4c7d-be80-4a6cebda905f","Type":"ContainerStarted","Data":"24e8290e62948ae4f6860c4d7e7a76c702452bba2df8967db5d0f9bc2a1fb3d9"} Jul 10 02:40:40.342529 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.342505 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/0.log" Jul 10 02:40:40.342615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.342545 2585 generic.go:356] "Generic (PLEG): container finished" podID="8e54d5b6-b253-4343-a8b0-8c22d69bec95" containerID="cb94c83a9239eabf76703a562fb0af7e1b99f9b1bf70f7ec1b8f9dfcd77a2170" exitCode=255 Jul 10 02:40:40.342675 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.342614 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xtwmx" event={"ID":"8e54d5b6-b253-4343-a8b0-8c22d69bec95","Type":"ContainerDied","Data":"cb94c83a9239eabf76703a562fb0af7e1b99f9b1bf70f7ec1b8f9dfcd77a2170"} Jul 10 02:40:40.342823 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.342807 2585 scope.go:117] "RemoveContainer" containerID="cb94c83a9239eabf76703a562fb0af7e1b99f9b1bf70f7ec1b8f9dfcd77a2170" Jul 10 02:40:40.353403 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.353349 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" podStartSLOduration=1.902328572 podStartE2EDuration="7.353336095s" totalImagesPullingTime="5.451007523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:40.352253593 +0000 UTC m=+47.061062133" watchObservedRunningTime="2026-07-10 02:40:40.353336095 +0000 UTC m=+47.062144656" Jul 10 02:40:40.384800 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.384685 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" podStartSLOduration=1.960697214 podStartE2EDuration="7.384640628s" totalImagesPullingTime="5.423943414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:40.366379408 +0000 UTC m=+47.075187957" watchObservedRunningTime="2026-07-10 02:40:40.384640628 +0000 UTC m=+47.093449165" Jul 10 02:40:40.404947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.404897 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-8z5qf" podStartSLOduration=1.91097941 podStartE2EDuration="7.404876433s" totalImagesPullingTime="5.493897023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:40.385713157 +0000 UTC m=+47.094521694" watchObservedRunningTime="2026-07-10 02:40:40.404876433 +0000 UTC m=+47.113684970" Jul 10 02:40:40.803939 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.803901 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:40.908797 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.908758 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:40.923636 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.923598 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c"] Jul 10 02:40:40.930627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.930593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:40.930814 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:40.930792 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:40:40.930875 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:40.930815 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-547685df6b-b8cpb: secret "image-registry-tls" not found Jul 10 02:40:40.930927 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:40.930897 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls podName:d542f2a5-c5fe-4e93-b2e7-3209085724c3 nodeName:}" failed. No retries permitted until 2026-07-10 02:40:48.930876748 +0000 UTC m=+55.639685268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls") pod "image-registry-547685df6b-b8cpb" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3") : secret "image-registry-tls" not found Jul 10 02:40:40.951450 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.951413 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c"] Jul 10 02:40:40.951622 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.951551 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" Jul 10 02:40:40.954413 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.954387 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-sfcgc\"" Jul 10 02:40:40.954624 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.954610 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 02:40:40.955628 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.955611 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 02:40:40.979094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:40.979062 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:41.031303 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.031268 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqfcz\" (UniqueName: \"kubernetes.io/projected/f0c93225-a245-4389-8f5b-86810e3ac838-kube-api-access-dqfcz\") pod \"migrator-7f59b54f6c-ckf8c\" (UID: \"f0c93225-a245-4389-8f5b-86810e3ac838\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" Jul 10 02:40:41.033903 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.033865 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:41.070851 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.070771 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jktcm_f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd/serve-healthcheck-canary/0.log" Jul 10 02:40:41.132269 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.132234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqfcz\" (UniqueName: \"kubernetes.io/projected/f0c93225-a245-4389-8f5b-86810e3ac838-kube-api-access-dqfcz\") pod \"migrator-7f59b54f6c-ckf8c\" (UID: \"f0c93225-a245-4389-8f5b-86810e3ac838\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" Jul 10 02:40:41.140600 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.140575 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqfcz\" (UniqueName: \"kubernetes.io/projected/f0c93225-a245-4389-8f5b-86810e3ac838-kube-api-access-dqfcz\") pod \"migrator-7f59b54f6c-ckf8c\" (UID: \"f0c93225-a245-4389-8f5b-86810e3ac838\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" Jul 10 02:40:41.237198 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.237167 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:41.260335 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.260312 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" Jul 10 02:40:41.294785 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.294751 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:41.347866 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.347839 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:40:41.348360 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.348342 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/0.log" Jul 10 02:40:41.348432 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.348386 2585 generic.go:356] "Generic (PLEG): container finished" podID="8e54d5b6-b253-4343-a8b0-8c22d69bec95" containerID="790843a6b9fba626635075fd1a4ff6fcc8105d0a55b188a80a35ae52224720be" exitCode=255 Jul 10 02:40:41.348514 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.348490 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xtwmx" event={"ID":"8e54d5b6-b253-4343-a8b0-8c22d69bec95","Type":"ContainerDied","Data":"790843a6b9fba626635075fd1a4ff6fcc8105d0a55b188a80a35ae52224720be"} Jul 10 02:40:41.348570 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.348563 2585 scope.go:117] "RemoveContainer" containerID="cb94c83a9239eabf76703a562fb0af7e1b99f9b1bf70f7ec1b8f9dfcd77a2170" Jul 10 02:40:41.348827 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.348808 2585 scope.go:117] "RemoveContainer" containerID="790843a6b9fba626635075fd1a4ff6fcc8105d0a55b188a80a35ae52224720be" Jul 10 02:40:41.349060 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:41.349027 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-xtwmx_openshift-console-operator(8e54d5b6-b253-4343-a8b0-8c22d69bec95)\"" pod="openshift-console-operator/console-operator-7866497494-xtwmx" podUID="8e54d5b6-b253-4343-a8b0-8c22d69bec95" Jul 10 02:40:41.400133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.400109 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c"] Jul 10 02:40:41.402196 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:41.402168 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0c93225_a245_4389_8f5b_86810e3ac838.slice/crio-f882beb0f4c4f61ec406145d545b0a5f8d3bb5ef6253a4fd218567b52c5e4932 WatchSource:0}: Error finding container f882beb0f4c4f61ec406145d545b0a5f8d3bb5ef6253a4fd218567b52c5e4932: Status 404 returned error can't find the container with id f882beb0f4c4f61ec406145d545b0a5f8d3bb5ef6253a4fd218567b52c5e4932 Jul 10 02:40:41.435715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.435683 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:41.435871 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.435742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:41.435871 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:41.435851 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:40:41.435989 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:41.435920 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:45.435901984 +0000 UTC m=+52.144710518 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : secret "insights-runtime-extractor-tls" not found Jul 10 02:40:41.436338 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.436310 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fdce4d-a91e-46e1-9326-0c151298684b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:41.535128 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.535094 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:41.603279 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:41.603193 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:42.353715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:42.353688 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:40:42.354137 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:42.354070 2585 scope.go:117] "RemoveContainer" containerID="790843a6b9fba626635075fd1a4ff6fcc8105d0a55b188a80a35ae52224720be" Jul 10 02:40:42.354272 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:42.354250 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-xtwmx_openshift-console-operator(8e54d5b6-b253-4343-a8b0-8c22d69bec95)\"" pod="openshift-console-operator/console-operator-7866497494-xtwmx" podUID="8e54d5b6-b253-4343-a8b0-8c22d69bec95" Jul 10 02:40:42.354996 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:42.354973 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" event={"ID":"f0c93225-a245-4389-8f5b-86810e3ac838","Type":"ContainerStarted","Data":"f882beb0f4c4f61ec406145d545b0a5f8d3bb5ef6253a4fd218567b52c5e4932"} Jul 10 02:40:43.151689 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.151637 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:43.205195 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.205165 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:43.359771 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.359678 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" event={"ID":"f0c93225-a245-4389-8f5b-86810e3ac838","Type":"ContainerStarted","Data":"e07df9055103b4b1a95a37c80a1d41713031a9a029525a5cd679e2cfafaf7a0a"} Jul 10 02:40:43.359771 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.359767 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" event={"ID":"f0c93225-a245-4389-8f5b-86810e3ac838","Type":"ContainerStarted","Data":"7f8b027eb6a342f9775c6b517bb0eddc9e760d68c5bc0a7265f2ef9accb7d6f2"} Jul 10 02:40:43.375731 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.375681 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-ckf8c" podStartSLOduration=1.683007146 podStartE2EDuration="3.375641133s" totalImagesPullingTime="1.692633987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:43.374992455 +0000 UTC m=+50.083800990" watchObservedRunningTime="2026-07-10 02:40:43.375641133 +0000 UTC m=+50.084449669" Jul 10 02:40:43.449233 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.449193 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:43.470141 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.470105 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:43.470305 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.470171 2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:43.470544 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.470527 2585 scope.go:117] "RemoveContainer" containerID="790843a6b9fba626635075fd1a4ff6fcc8105d0a55b188a80a35ae52224720be" Jul 10 02:40:43.470723 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:43.470703 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-xtwmx_openshift-console-operator(8e54d5b6-b253-4343-a8b0-8c22d69bec95)\"" pod="openshift-console-operator/console-operator-7866497494-xtwmx" podUID="8e54d5b6-b253-4343-a8b0-8c22d69bec95" Jul 10 02:40:43.549605 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.549569 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:43.954301 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:43.954255 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:43.954479 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:43.954402 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 02:40:43.954479 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:43.954469 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls podName:33303bd9-1b00-4884-a4bb-028e5620fafe nodeName:}" failed. No retries permitted until 2026-07-10 02:40:59.954452129 +0000 UTC m=+66.663260644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls") pod "dns-default-4w9w5" (UID: "33303bd9-1b00-4884-a4bb-028e5620fafe") : secret "dns-default-metrics-tls" not found Jul 10 02:40:44.078479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:44.078442 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:45.466829 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:45.466791 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:45.467234 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:45.466957 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 02:40:45.467234 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:45.467044 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls podName:05fdce4d-a91e-46e1-9326-0c151298684b nodeName:}" failed. No retries permitted until 2026-07-10 02:40:53.467027854 +0000 UTC m=+60.175836373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7vr8g" (UID: "05fdce4d-a91e-46e1-9326-0c151298684b") : secret "insights-runtime-extractor-tls" not found Jul 10 02:40:46.714151 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:46.714113 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:47.068819 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:47.068736 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:47.203819 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:47.203778 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:47.264772 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:47.264735 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:48.454528 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:48.454491 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:48.682699 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:48.682663 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:48.778509 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:48.778423 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:48.877573 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:48.877538 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:48.978546 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:48.978511 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:48.992663 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:48.992623 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:40:48.992796 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:48.992771 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 02:40:48.992796 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:48.992787 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-547685df6b-b8cpb: secret "image-registry-tls" not found Jul 10 02:40:48.992889 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:40:48.992840 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls podName:d542f2a5-c5fe-4e93-b2e7-3209085724c3 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:04.992824151 +0000 UTC m=+71.701632666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls") pod "image-registry-547685df6b-b8cpb" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3") : secret "image-registry-tls" not found Jul 10 02:40:49.157425 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:49.157343 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:50.102361 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:50.102330 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:50.203032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:50.203001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:50.362346 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:50.362249 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:50.767839 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:50.767790 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:50.963580 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:50.963543 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:51.552266 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:51.552205 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:51.989844 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:51.989806 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:52.089615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:52.089574 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:52.190876 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:52.190828 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:52.759816 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:52.759777 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:53.158415 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.158333 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:53.275760 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.275731 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bvvqn" Jul 10 02:40:53.357404 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.357371 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:53.529909 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.529873 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:53.532302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.532264 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fdce4d-a91e-46e1-9326-0c151298684b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7vr8g\" (UID: \"05fdce4d-a91e-46e1-9326-0c151298684b\") " pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:53.553224 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.553191 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:53.616866 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.616827 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7vr8g" Jul 10 02:40:53.769307 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:53.769280 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7vr8g"] Jul 10 02:40:53.772147 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:40:53.772124 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fdce4d_a91e_46e1_9326_0c151298684b.slice/crio-7c6ba67328e5b82e9fd0fc79123c07459d4250190c76e60d821181bf0c99518a WatchSource:0}: Error finding container 7c6ba67328e5b82e9fd0fc79123c07459d4250190c76e60d821181bf0c99518a: Status 404 returned error can't find the container with id 7c6ba67328e5b82e9fd0fc79123c07459d4250190c76e60d821181bf0c99518a Jul 10 02:40:54.396906 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:54.396865 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7vr8g" event={"ID":"05fdce4d-a91e-46e1-9326-0c151298684b","Type":"ContainerStarted","Data":"7c6ba67328e5b82e9fd0fc79123c07459d4250190c76e60d821181bf0c99518a"} Jul 10 02:40:54.557000 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:54.556944 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:54.664004 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:54.663921 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:54.961366 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:54.961196 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:55.160732 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:55.160693 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:55.359212 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:55.359135 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:55.558470 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:55.558414 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:56.106114 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:56.106074 2585 scope.go:117] "RemoveContainer" containerID="790843a6b9fba626635075fd1a4ff6fcc8105d0a55b188a80a35ae52224720be" Jul 10 02:40:56.558300 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:56.558266 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:56.754181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:56.754136 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:57.154387 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:57.154354 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:57.356525 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:57.356447 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:57.954456 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:57.954410 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.199484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.199457 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.300989 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.300904 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.402040 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.402003 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.412011 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.411987 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:40:58.412137 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.412084 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-xtwmx" event={"ID":"8e54d5b6-b253-4343-a8b0-8c22d69bec95","Type":"ContainerStarted","Data":"4be8e017cd8adf80ecfbc4245a99fcce327373125c0ad74988633968ef3e7c9f"} Jul 10 02:40:58.412483 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.412462 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:58.414035 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.414015 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7vr8g" event={"ID":"05fdce4d-a91e-46e1-9326-0c151298684b","Type":"ContainerStarted","Data":"df87dd1991b64393706e74cf4c3d7efec753dbdc2e5c9937d24871972ba18302"} Jul 10 02:40:58.414133 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.414059 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7vr8g" event={"ID":"05fdce4d-a91e-46e1-9326-0c151298684b","Type":"ContainerStarted","Data":"b2b90068d6c821671c45c85aff215150ec1582b296377d12acd8faf093535917"} Jul 10 02:40:58.417326 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.417308 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-xtwmx" Jul 10 02:40:58.430721 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.430685 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-xtwmx" podStartSLOduration=19.861502409 podStartE2EDuration="25.430673282s" totalImagesPullingTime="5.569170873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:40:58.429270683 +0000 UTC m=+65.138079219" watchObservedRunningTime="2026-07-10 02:40:58.430673282 +0000 UTC m=+65.139481815" Jul 10 02:40:58.499453 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.499421 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.680764 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.680719 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.780865 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.780832 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:58.956328 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:58.956241 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:59.071161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.071108 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:59.179026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.178990 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:59.331404 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.331309 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:59.433112 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.433075 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:59.580556 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.580518 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:59.583626 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.583563 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 02:40:59.593389 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.593361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53b3aa6-f527-411f-ad5d-0f4bd076dd0b-metrics-certs\") pod \"network-metrics-daemon-zn6fq\" (UID: \"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b\") " pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:59.636949 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.636917 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jw76v\"" Jul 10 02:40:59.644366 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.644332 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zn6fq" Jul 10 02:40:59.681781 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.681744 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:59.681970 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.681782 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:59.684715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.684690 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 02:40:59.684856 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.684827 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxjr8\" (UniqueName: \"kubernetes.io/projected/1eddeeef-12e8-4c5f-b692-5f52c8642bbe-kube-api-access-mxjr8\") pod \"network-check-target-5wkwz\" (UID: \"1eddeeef-12e8-4c5f-b692-5f52c8642bbe\") " pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:59.695360 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.695334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0991902f-cbfb-41d6-9399-519aac9b646b-original-pull-secret\") pod \"global-pull-secret-syncer-wd6fm\" (UID: \"0991902f-cbfb-41d6-9399-519aac9b646b\") " pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:59.882672 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.882550 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:40:59.918759 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.918723 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wd6fm" Jul 10 02:40:59.929889 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.929862 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9pgv\"" Jul 10 02:40:59.937260 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.937236 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:40:59.983438 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.983402 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:40:59.985996 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:40:59.985968 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33303bd9-1b00-4884-a4bb-028e5620fafe-metrics-tls\") pod \"dns-default-4w9w5\" (UID: \"33303bd9-1b00-4884-a4bb-028e5620fafe\") " pod="openshift-dns/dns-default-4w9w5" Jul 10 02:41:00.082389 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.082336 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:00.246342 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.246252 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8l52l\"" Jul 10 02:41:00.246588 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.246422 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:00.254634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.254606 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4w9w5" Jul 10 02:41:00.804874 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.804847 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5wkwz"] Jul 10 02:41:00.822608 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.822585 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4w9w5"] Jul 10 02:41:00.844415 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.844390 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wd6fm"] Jul 10 02:41:00.861688 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:00.861664 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zn6fq"] Jul 10 02:41:01.004627 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:01.004582 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33303bd9_1b00_4884_a4bb_028e5620fafe.slice/crio-cced63a593c6248db37f901c1284d6b48bdab5ea1876b949f656549ea1de8b55 WatchSource:0}: Error finding container cced63a593c6248db37f901c1284d6b48bdab5ea1876b949f656549ea1de8b55: Status 404 returned error can't find the container with id cced63a593c6248db37f901c1284d6b48bdab5ea1876b949f656549ea1de8b55 Jul 10 02:41:01.005124 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:01.005093 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0991902f_cbfb_41d6_9399_519aac9b646b.slice/crio-c09e92b599b64e6184fe09d7eaf6a7e89dc48a1df9236d7292d75231cc9f628d WatchSource:0}: Error finding container c09e92b599b64e6184fe09d7eaf6a7e89dc48a1df9236d7292d75231cc9f628d: Status 404 returned error can't find the container with id c09e92b599b64e6184fe09d7eaf6a7e89dc48a1df9236d7292d75231cc9f628d Jul 10 02:41:01.006015 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:01.005976 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53b3aa6_f527_411f_ad5d_0f4bd076dd0b.slice/crio-26a3dd6cd3ed80b64123532f82227a4ab22994d185f5e0ecfeb6956a126d9308 WatchSource:0}: Error finding container 26a3dd6cd3ed80b64123532f82227a4ab22994d185f5e0ecfeb6956a126d9308: Status 404 returned error can't find the container with id 26a3dd6cd3ed80b64123532f82227a4ab22994d185f5e0ecfeb6956a126d9308 Jul 10 02:41:01.121885 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.121858 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:01.218029 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.217978 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:01.426634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.426599 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4w9w5" event={"ID":"33303bd9-1b00-4884-a4bb-028e5620fafe","Type":"ContainerStarted","Data":"cced63a593c6248db37f901c1284d6b48bdab5ea1876b949f656549ea1de8b55"} Jul 10 02:41:01.428132 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.428108 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5wkwz" event={"ID":"1eddeeef-12e8-4c5f-b692-5f52c8642bbe","Type":"ContainerStarted","Data":"160fb234bc0a02b8ed685aeb2c9de7eb4e42ce09ce0155b6680fbd2e24fe6a2e"} Jul 10 02:41:01.428251 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.428157 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5wkwz" event={"ID":"1eddeeef-12e8-4c5f-b692-5f52c8642bbe","Type":"ContainerStarted","Data":"07f575fd127a8d328fc430513e032a92f0be51966b753dc7d840df9c274df55d"} Jul 10 02:41:01.428251 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.428185 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:41:01.430102 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.430079 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7vr8g" event={"ID":"05fdce4d-a91e-46e1-9326-0c151298684b","Type":"ContainerStarted","Data":"b8560d26010f76f696c920e99f9ac9ce0b2c5e5c1a4be5b1ee86abddbddc9546"} Jul 10 02:41:01.431110 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.431093 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wd6fm" event={"ID":"0991902f-cbfb-41d6-9399-519aac9b646b","Type":"ContainerStarted","Data":"c09e92b599b64e6184fe09d7eaf6a7e89dc48a1df9236d7292d75231cc9f628d"} Jul 10 02:41:01.432197 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.432167 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zn6fq" event={"ID":"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b","Type":"ContainerStarted","Data":"26a3dd6cd3ed80b64123532f82227a4ab22994d185f5e0ecfeb6956a126d9308"} Jul 10 02:41:01.445562 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.445500 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5wkwz" podStartSLOduration=67.44548699 podStartE2EDuration="1m7.44548699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:01.444575478 +0000 UTC m=+68.153384013" watchObservedRunningTime="2026-07-10 02:41:01.44548699 +0000 UTC m=+68.154295526" Jul 10 02:41:01.462988 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:01.462949 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7vr8g" podStartSLOduration=17.315990608 podStartE2EDuration="24.462936297s" totalImagesPullingTime="7.146945689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:40:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:01.461789801 +0000 UTC m=+68.170598339" watchObservedRunningTime="2026-07-10 02:41:01.462936297 +0000 UTC m=+68.171744882" Jul 10 02:41:03.442485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.442447 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zn6fq" event={"ID":"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b","Type":"ContainerStarted","Data":"821461c901f1c520d1c546b8cef3824577f765adeeb0715e27cc093343643c36"} Jul 10 02:41:03.442851 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.442516 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zn6fq" event={"ID":"f53b3aa6-f527-411f-ad5d-0f4bd076dd0b","Type":"ContainerStarted","Data":"a37a1eb76847505539b1bbbd5fec82a2d9f02ab05d0a5bfc9c70e1c867317b00"} Jul 10 02:41:03.444666 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.444628 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4w9w5" event={"ID":"33303bd9-1b00-4884-a4bb-028e5620fafe","Type":"ContainerStarted","Data":"093af42ec5f83acd5c84d5da69b17afa2806489e0c6434fba439d7037c335e83"} Jul 10 02:41:03.444790 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.444685 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4w9w5" event={"ID":"33303bd9-1b00-4884-a4bb-028e5620fafe","Type":"ContainerStarted","Data":"db72fb9bdb58b14ffb4eca28f23943e86e4542ae4784fed2d35d02735de3dbca"} Jul 10 02:41:03.444790 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.444715 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4w9w5" Jul 10 02:41:03.461433 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.461336 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zn6fq" podStartSLOduration=67.299873682 podStartE2EDuration="1m9.461315842s" totalImagesPullingTime="2.16144216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:03.460144737 +0000 UTC m=+70.168953273" watchObservedRunningTime="2026-07-10 02:41:03.461315842 +0000 UTC m=+70.170124380" Jul 10 02:41:03.478111 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:03.478040 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4w9w5" podStartSLOduration=33.311261289 podStartE2EDuration="35.478018759s" totalImagesPullingTime="2.16675747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:03.475992893 +0000 UTC m=+70.184801429" watchObservedRunningTime="2026-07-10 02:41:03.478018759 +0000 UTC m=+70.186827296" Jul 10 02:41:05.028302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.028266 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:05.030881 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.030853 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"image-registry-547685df6b-b8cpb\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:05.171477 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.171377 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:05.272699 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.272643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:05.283060 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.283034 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w6ngk\"" Jul 10 02:41:05.291367 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.291345 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:05.418203 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.418180 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-547685df6b-b8cpb"] Jul 10 02:41:05.420473 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:05.420440 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd542f2a5_c5fe_4e93_b2e7_3209085724c3.slice/crio-2e2b3fe85faa9ebdb4f85770d9478bbcbdc5f77abacfc783767f93e3b8a9d090 WatchSource:0}: Error finding container 2e2b3fe85faa9ebdb4f85770d9478bbcbdc5f77abacfc783767f93e3b8a9d090: Status 404 returned error can't find the container with id 2e2b3fe85faa9ebdb4f85770d9478bbcbdc5f77abacfc783767f93e3b8a9d090 Jul 10 02:41:05.453879 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.453843 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" event={"ID":"d542f2a5-c5fe-4e93-b2e7-3209085724c3","Type":"ContainerStarted","Data":"2e2b3fe85faa9ebdb4f85770d9478bbcbdc5f77abacfc783767f93e3b8a9d090"} Jul 10 02:41:05.455629 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.455474 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wd6fm" event={"ID":"0991902f-cbfb-41d6-9399-519aac9b646b","Type":"ContainerStarted","Data":"fd6ad9ef8a469a84c2945fa456d0880d1ca5e98140609b0197eeff118f65f6f7"} Jul 10 02:41:05.494093 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:05.494043 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wd6fm" podStartSLOduration=67.620922753 podStartE2EDuration="1m11.494026985s" totalImagesPullingTime="3.873104232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:05.493279618 +0000 UTC m=+72.202088154" watchObservedRunningTime="2026-07-10 02:41:05.494026985 +0000 UTC m=+72.202835574" Jul 10 02:41:06.460689 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.460639 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" event={"ID":"d542f2a5-c5fe-4e93-b2e7-3209085724c3","Type":"ContainerStarted","Data":"4adbb2de93c875d076e64acd847a551e2ee2d206d957b872c80d36ac8195c966"} Jul 10 02:41:06.461269 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.460819 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:06.488109 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.488047 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-547685df6b-b8cpb"] Jul 10 02:41:06.509269 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.509202 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" podStartSLOduration=33.509177535 podStartE2EDuration="33.509177535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:06.508297265 +0000 UTC m=+73.217105801" watchObservedRunningTime="2026-07-10 02:41:06.509177535 +0000 UTC m=+73.217986072" Jul 10 02:41:06.598725 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.598688 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d5499fc78-n2xpq"] Jul 10 02:41:06.602054 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.602026 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.602573 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.602546 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-457qp"] Jul 10 02:41:06.604427 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.604404 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:06.606612 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.606583 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:06.607262 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.607232 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-q6jjm\"" Jul 10 02:41:06.607549 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.607534 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 02:41:06.607810 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.607793 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 02:41:06.619997 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.619971 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d5499fc78-n2xpq"] Jul 10 02:41:06.625810 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.625782 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-457qp"] Jul 10 02:41:06.710208 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.710160 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:06.743672 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743582 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj9pn\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-kube-api-access-jj9pn\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.743672 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-bound-sa-token\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.743812 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3e24133-eb82-45ae-a1a0-534d64a4528d-installation-pull-secrets\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.743812 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743707 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcx2j\" (UniqueName: \"kubernetes.io/projected/5f80bd45-9108-40f6-b232-82b2ad2e4b03-kube-api-access-lcx2j\") pod \"downloads-68c9cbb497-457qp\" (UID: \"5f80bd45-9108-40f6-b232-82b2ad2e4b03\") " pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:06.743812 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-registry-tls\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.743812 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743774 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3e24133-eb82-45ae-a1a0-534d64a4528d-registry-certificates\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.743812 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743796 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3e24133-eb82-45ae-a1a0-534d64a4528d-image-registry-private-configuration\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.744037 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743818 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3e24133-eb82-45ae-a1a0-534d64a4528d-ca-trust-extracted\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.744037 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.743836 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3e24133-eb82-45ae-a1a0-534d64a4528d-trusted-ca\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.805806 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.805774 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:06.844587 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcx2j\" (UniqueName: \"kubernetes.io/projected/5f80bd45-9108-40f6-b232-82b2ad2e4b03-kube-api-access-lcx2j\") pod \"downloads-68c9cbb497-457qp\" (UID: \"5f80bd45-9108-40f6-b232-82b2ad2e4b03\") " pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:06.844807 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844598 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-registry-tls\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.844807 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844620 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3e24133-eb82-45ae-a1a0-534d64a4528d-registry-certificates\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.844807 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3e24133-eb82-45ae-a1a0-534d64a4528d-image-registry-private-configuration\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.844807 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844753 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3e24133-eb82-45ae-a1a0-534d64a4528d-ca-trust-extracted\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.844807 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844789 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3e24133-eb82-45ae-a1a0-534d64a4528d-trusted-ca\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.845117 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844927 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj9pn\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-kube-api-access-jj9pn\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.845117 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.844976 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-bound-sa-token\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.845117 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.845010 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3e24133-eb82-45ae-a1a0-534d64a4528d-installation-pull-secrets\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.845273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.845222 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3e24133-eb82-45ae-a1a0-534d64a4528d-ca-trust-extracted\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.845559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.845541 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3e24133-eb82-45ae-a1a0-534d64a4528d-registry-certificates\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.845872 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.845849 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3e24133-eb82-45ae-a1a0-534d64a4528d-trusted-ca\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.847295 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.847276 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-registry-tls\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.847384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.847300 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3e24133-eb82-45ae-a1a0-534d64a4528d-image-registry-private-configuration\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.847384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.847334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3e24133-eb82-45ae-a1a0-534d64a4528d-installation-pull-secrets\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.856267 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.856239 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-bound-sa-token\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.856347 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.856336 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcx2j\" (UniqueName: \"kubernetes.io/projected/5f80bd45-9108-40f6-b232-82b2ad2e4b03-kube-api-access-lcx2j\") pod \"downloads-68c9cbb497-457qp\" (UID: \"5f80bd45-9108-40f6-b232-82b2ad2e4b03\") " pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:06.857267 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.857249 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj9pn\" (UniqueName: \"kubernetes.io/projected/b3e24133-eb82-45ae-a1a0-534d64a4528d-kube-api-access-jj9pn\") pod \"image-registry-5d5499fc78-n2xpq\" (UID: \"b3e24133-eb82-45ae-a1a0-534d64a4528d\") " pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.900575 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.900537 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:06.913467 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.913438 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:06.918323 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:06.918308 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:07.054736 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.054531 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d5499fc78-n2xpq"] Jul 10 02:41:07.057284 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:07.057245 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e24133_eb82_45ae_a1a0_534d64a4528d.slice/crio-a83bb673a5b0062fe64165ce312437752df6aa6bd4625eb57b1701cf03abc2e3 WatchSource:0}: Error finding container a83bb673a5b0062fe64165ce312437752df6aa6bd4625eb57b1701cf03abc2e3: Status 404 returned error can't find the container with id a83bb673a5b0062fe64165ce312437752df6aa6bd4625eb57b1701cf03abc2e3 Jul 10 02:41:07.070887 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.070861 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-457qp"] Jul 10 02:41:07.073752 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:07.073727 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f80bd45_9108_40f6_b232_82b2ad2e4b03.slice/crio-8f5dd37abab00983e5bdf13c2660b1c39216007ee3db500b8a8518cbec316d13 WatchSource:0}: Error finding container 8f5dd37abab00983e5bdf13c2660b1c39216007ee3db500b8a8518cbec316d13: Status 404 returned error can't find the container with id 8f5dd37abab00983e5bdf13c2660b1c39216007ee3db500b8a8518cbec316d13 Jul 10 02:41:07.465627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.465595 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-457qp" event={"ID":"5f80bd45-9108-40f6-b232-82b2ad2e4b03","Type":"ContainerStarted","Data":"8f5dd37abab00983e5bdf13c2660b1c39216007ee3db500b8a8518cbec316d13"} Jul 10 02:41:07.467243 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.467206 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" event={"ID":"b3e24133-eb82-45ae-a1a0-534d64a4528d","Type":"ContainerStarted","Data":"dc68b70532afdea7a411f23bf8eb33a6a279d022120b3c21378e24f01182eeb6"} Jul 10 02:41:07.467395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.467289 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:07.467395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.467330 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" event={"ID":"b3e24133-eb82-45ae-a1a0-534d64a4528d","Type":"ContainerStarted","Data":"a83bb673a5b0062fe64165ce312437752df6aa6bd4625eb57b1701cf03abc2e3"} Jul 10 02:41:07.489365 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.489312 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" podStartSLOduration=1.489296257 podStartE2EDuration="1.489296257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:07.489020144 +0000 UTC m=+74.197828672" watchObservedRunningTime="2026-07-10 02:41:07.489296257 +0000 UTC m=+74.198104794" Jul 10 02:41:07.720532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:07.720433 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:08.118906 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:08.118831 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:08.195189 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:08.195149 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:08.294013 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:08.293981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:08.581214 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:08.581174 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:08.905254 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:08.905176 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:09.005969 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:09.005935 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:09.318416 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:09.318373 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:09.419008 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:09.418959 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:10.329438 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.329402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:10.439855 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.439820 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:10.542280 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.542228 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:10.914636 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.914601 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b944799f-2n76z"] Jul 10 02:41:10.953189 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.953157 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b944799f-2n76z"] Jul 10 02:41:10.953359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.953286 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:10.956511 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.956476 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sfgqp\"" Jul 10 02:41:10.956671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.956534 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 02:41:10.956671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.956577 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 02:41:10.956860 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.956477 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 02:41:10.956860 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.956750 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 02:41:10.957578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:10.957541 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 02:41:11.079014 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.078978 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-service-ca\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.079014 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.079019 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-console-config\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.079247 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.079035 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-oauth-config\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.079247 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.079064 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-serving-cert\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.079247 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.079089 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8mbq\" (UniqueName: \"kubernetes.io/projected/bc18857c-a776-40a6-a883-780c418b127a-kube-api-access-w8mbq\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.079247 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.079145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-oauth-serving-cert\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.180266 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.180231 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-oauth-config\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.180266 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.180273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-serving-cert\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.180485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.180303 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mbq\" (UniqueName: \"kubernetes.io/projected/bc18857c-a776-40a6-a883-780c418b127a-kube-api-access-w8mbq\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.180485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.180362 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-oauth-serving-cert\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.180485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.180403 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-service-ca\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.180485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.180429 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-console-config\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.181253 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.181219 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-oauth-serving-cert\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.181253 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.181230 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-console-config\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.181432 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.181323 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-service-ca\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.183110 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.183084 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-oauth-config\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.183235 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.183084 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-serving-cert\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.188779 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.188759 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8mbq\" (UniqueName: \"kubernetes.io/projected/bc18857c-a776-40a6-a883-780c418b127a-kube-api-access-w8mbq\") pod \"console-b944799f-2n76z\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.265709 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.265425 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:11.439148 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.439125 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b944799f-2n76z"] Jul 10 02:41:11.443054 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:11.443022 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc18857c_a776_40a6_a883_780c418b127a.slice/crio-32ceebb4ea2980942c7a1ab8df8d94daae30ebfff02d26785a74582880e1463d WatchSource:0}: Error finding container 32ceebb4ea2980942c7a1ab8df8d94daae30ebfff02d26785a74582880e1463d: Status 404 returned error can't find the container with id 32ceebb4ea2980942c7a1ab8df8d94daae30ebfff02d26785a74582880e1463d Jul 10 02:41:11.485264 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:11.485225 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b944799f-2n76z" event={"ID":"bc18857c-a776-40a6-a883-780c418b127a","Type":"ContainerStarted","Data":"32ceebb4ea2980942c7a1ab8df8d94daae30ebfff02d26785a74582880e1463d"} Jul 10 02:41:12.325688 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:12.325602 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:12.914323 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:12.914271 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:12.970990 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:12.970910 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:13.451622 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.451592 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4w9w5" Jul 10 02:41:13.494216 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.494139 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:13.589053 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.589009 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:13.720548 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.720418 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:13.724110 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.724079 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz"] Jul 10 02:41:13.726901 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.726867 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.729597 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.729574 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 02:41:13.729905 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.729888 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 02:41:13.730117 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.730100 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 02:41:13.730341 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.730321 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 02:41:13.730518 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.730500 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6zpv6\"" Jul 10 02:41:13.730743 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.730719 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 02:41:13.738376 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.738347 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz"] Jul 10 02:41:13.775338 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.774695 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qlsbc"] Jul 10 02:41:13.778195 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.778094 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.780821 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.780798 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 02:41:13.780960 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.780817 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 02:41:13.780960 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.780874 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 02:41:13.781222 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.781150 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dlwvz\"" Jul 10 02:41:13.803619 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.803538 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ng2\" (UniqueName: \"kubernetes.io/projected/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-kube-api-access-j6ng2\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.804129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.803589 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.804129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.803915 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.804129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.803968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.808310 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.808261 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:13.905380 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-wtmp\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905380 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905345 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905601 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905440 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-root\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905601 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-sys\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905601 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905558 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.905601 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905597 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905630 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ng2\" (UniqueName: \"kubernetes.io/projected/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-kube-api-access-j6ng2\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905699 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6059534-5d75-44fe-8288-fb02f5ec5e88-metrics-client-ca\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905731 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74csr\" (UniqueName: \"kubernetes.io/projected/b6059534-5d75-44fe-8288-fb02f5ec5e88-kube-api-access-74csr\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:13.905740 2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905774 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.905863 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:13.905800 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls podName:4e8bcddc-4610-4ab6-b030-eb1fbf49fd92 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:14.405781171 +0000 UTC m=+81.114589690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-d9nwz" (UID: "4e8bcddc-4610-4ab6-b030-eb1fbf49fd92") : secret "openshift-state-metrics-tls" not found Jul 10 02:41:13.906200 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905944 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-textfile\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.906200 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.905981 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-tls\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:13.906466 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.906439 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.908610 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.908566 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:13.914106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:13.914082 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ng2\" (UniqueName: \"kubernetes.io/projected/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-kube-api-access-j6ng2\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:14.007479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-root\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.007479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007427 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-sys\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.007479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007461 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-root\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.007479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6059534-5d75-44fe-8288-fb02f5ec5e88-metrics-client-ca\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007498 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-sys\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007520 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74csr\" (UniqueName: \"kubernetes.io/projected/b6059534-5d75-44fe-8288-fb02f5ec5e88-kube-api-access-74csr\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007579 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-textfile\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007600 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-tls\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007634 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-wtmp\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.007681 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:14.007792 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:14.007879 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-tls podName:b6059534-5d75-44fe-8288-fb02f5ec5e88 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:14.50785894 +0000 UTC m=+81.216667453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-tls") pod "node-exporter-qlsbc" (UID: "b6059534-5d75-44fe-8288-fb02f5ec5e88") : secret "node-exporter-tls" not found Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.008177 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6059534-5d75-44fe-8288-fb02f5ec5e88-metrics-client-ca\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.008180 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-wtmp\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.008319 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-textfile\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.008774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.008607 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-accelerators-collector-config\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.010359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.010333 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.017439 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.017410 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74csr\" (UniqueName: \"kubernetes.io/projected/b6059534-5d75-44fe-8288-fb02f5ec5e88-kube-api-access-74csr\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.104336 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.104299 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:14.204568 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.204533 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:14.264083 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.263873 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:14.411431 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.411392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:14.411625 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:14.411575 2585 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 10 02:41:14.411718 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:14.411672 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls podName:4e8bcddc-4610-4ab6-b030-eb1fbf49fd92 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:15.411633745 +0000 UTC m=+82.120442264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-d9nwz" (UID: "4e8bcddc-4610-4ab6-b030-eb1fbf49fd92") : secret "openshift-state-metrics-tls" not found Jul 10 02:41:14.481892 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.481820 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:14.512705 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.512673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-tls\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.515820 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.515760 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6059534-5d75-44fe-8288-fb02f5ec5e88-node-exporter-tls\") pod \"node-exporter-qlsbc\" (UID: \"b6059534-5d75-44fe-8288-fb02f5ec5e88\") " pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.691322 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.691265 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qlsbc" Jul 10 02:41:14.733206 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.733049 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:14.773717 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:14.773682 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6059534_5d75_44fe_8288_fb02f5ec5e88.slice/crio-b2426c5fbc21fcda7a7b79d649a27a0725d4868230b58556c092587a6c772e17 WatchSource:0}: Error finding container b2426c5fbc21fcda7a7b79d649a27a0725d4868230b58556c092587a6c772e17: Status 404 returned error can't find the container with id b2426c5fbc21fcda7a7b79d649a27a0725d4868230b58556c092587a6c772e17 Jul 10 02:41:14.818570 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:14.818545 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:15.034882 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.034725 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:15.134592 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.134556 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:15.421898 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.421743 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:15.424359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.424333 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4e8bcddc-4610-4ab6-b030-eb1fbf49fd92-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-d9nwz\" (UID: \"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:15.505786 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.505744 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b944799f-2n76z" event={"ID":"bc18857c-a776-40a6-a883-780c418b127a","Type":"ContainerStarted","Data":"b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464"} Jul 10 02:41:15.507276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.507248 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlsbc" event={"ID":"b6059534-5d75-44fe-8288-fb02f5ec5e88","Type":"ContainerStarted","Data":"b2426c5fbc21fcda7a7b79d649a27a0725d4868230b58556c092587a6c772e17"} Jul 10 02:41:15.524689 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.524576 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b944799f-2n76z" podStartSLOduration=2.168988815 podStartE2EDuration="5.52456039s" totalImagesPullingTime="3.355571575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:15.524101388 +0000 UTC m=+82.232909925" watchObservedRunningTime="2026-07-10 02:41:15.52456039 +0000 UTC m=+82.233368927" Jul 10 02:41:15.539596 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.539561 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" Jul 10 02:41:15.691732 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.691696 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz"] Jul 10 02:41:15.818796 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:15.818760 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8bcddc_4610_4ab6_b030_eb1fbf49fd92.slice/crio-4ca1e9b1cb08f86e38129b4ecba8100f9c65f0b7a4aac665c3a0013ab2bfe3dd WatchSource:0}: Error finding container 4ca1e9b1cb08f86e38129b4ecba8100f9c65f0b7a4aac665c3a0013ab2bfe3dd: Status 404 returned error can't find the container with id 4ca1e9b1cb08f86e38129b4ecba8100f9c65f0b7a4aac665c3a0013ab2bfe3dd Jul 10 02:41:15.916505 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:15.916442 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.014240 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.014208 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.204139 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.204029 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.304480 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.304444 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.513802 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.513718 2585 generic.go:356] "Generic (PLEG): container finished" podID="b6059534-5d75-44fe-8288-fb02f5ec5e88" containerID="84be8a2cf5d4ae842c9ba484c9e03dc83245c50e3c72977adb2c0c2e8284bf09" exitCode=0 Jul 10 02:41:16.513961 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.513810 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlsbc" event={"ID":"b6059534-5d75-44fe-8288-fb02f5ec5e88","Type":"ContainerDied","Data":"84be8a2cf5d4ae842c9ba484c9e03dc83245c50e3c72977adb2c0c2e8284bf09"} Jul 10 02:41:16.516223 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.516190 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" event={"ID":"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92","Type":"ContainerStarted","Data":"aeb04a2b5248b74cb0661d95d926144399c86c4fe5db126e980c647ea8caed8d"} Jul 10 02:41:16.516329 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.516253 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" event={"ID":"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92","Type":"ContainerStarted","Data":"94bfc36e315553f5d099fc1684695e434d1c96e60eeb50ec7dd7052c02b34ed0"} Jul 10 02:41:16.516329 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.516282 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" event={"ID":"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92","Type":"ContainerStarted","Data":"4ca1e9b1cb08f86e38129b4ecba8100f9c65f0b7a4aac665c3a0013ab2bfe3dd"} Jul 10 02:41:16.690786 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.690751 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.753908 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.753873 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.874675 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.874575 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:16.999695 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:16.999661 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:17.522095 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:17.522057 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlsbc" event={"ID":"b6059534-5d75-44fe-8288-fb02f5ec5e88","Type":"ContainerStarted","Data":"a64fe9f32230f860ea07dbb4f1dfd00738e4bd67845bbd0349a4c349cfd5e781"} Jul 10 02:41:18.148102 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.148073 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-878d8cd45-7jsqw"] Jul 10 02:41:18.150343 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.150302 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.154183 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.154155 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 02:41:18.154183 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.154188 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e3kcun2g0hukm\"" Jul 10 02:41:18.154383 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.154269 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 02:41:18.154383 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.154158 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-hxp6h\"" Jul 10 02:41:18.154542 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.154406 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 02:41:18.154690 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.154674 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 02:41:18.161374 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.161350 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-878d8cd45-7jsqw"] Jul 10 02:41:18.247057 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247024 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-secret-metrics-server-client-certs\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.247057 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247057 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/84511177-2df9-477f-9cc7-2420c1500205-audit-log\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.247302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247091 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7s5\" (UniqueName: \"kubernetes.io/projected/84511177-2df9-477f-9cc7-2420c1500205-kube-api-access-xh7s5\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.247302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84511177-2df9-477f-9cc7-2420c1500205-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.247302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247232 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/84511177-2df9-477f-9cc7-2420c1500205-metrics-server-audit-profiles\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.247302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247261 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-client-ca-bundle\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.247487 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.247312 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-secret-metrics-server-tls\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.348753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348718 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/84511177-2df9-477f-9cc7-2420c1500205-metrics-server-audit-profiles\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.348950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348765 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-client-ca-bundle\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.348950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348814 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-secret-metrics-server-tls\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.348950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-secret-metrics-server-client-certs\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.348950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348874 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/84511177-2df9-477f-9cc7-2420c1500205-audit-log\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.348950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348903 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7s5\" (UniqueName: \"kubernetes.io/projected/84511177-2df9-477f-9cc7-2420c1500205-kube-api-access-xh7s5\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.349213 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.348956 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84511177-2df9-477f-9cc7-2420c1500205-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.349371 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.349353 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/84511177-2df9-477f-9cc7-2420c1500205-audit-log\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.349679 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.349633 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84511177-2df9-477f-9cc7-2420c1500205-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.349919 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.349894 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/84511177-2df9-477f-9cc7-2420c1500205-metrics-server-audit-profiles\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.351683 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.351604 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-client-ca-bundle\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.351791 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.351774 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-secret-metrics-server-client-certs\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.351857 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.351839 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/84511177-2df9-477f-9cc7-2420c1500205-secret-metrics-server-tls\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.357165 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.357129 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7s5\" (UniqueName: \"kubernetes.io/projected/84511177-2df9-477f-9cc7-2420c1500205-kube-api-access-xh7s5\") pod \"metrics-server-878d8cd45-7jsqw\" (UID: \"84511177-2df9-477f-9cc7-2420c1500205\") " pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.461600 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.461560 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:18.559311 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.559244 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:18.572286 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.572251 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz"] Jul 10 02:41:18.576192 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.576167 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b944799f-2n76z"] Jul 10 02:41:18.576350 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.576285 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.579042 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.579022 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 10 02:41:18.579169 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.579089 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jvt5w\"" Jul 10 02:41:18.583009 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.582988 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz"] Jul 10 02:41:18.597814 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.597787 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bcc8b9b77-htx9w"] Jul 10 02:41:18.600817 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.600794 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.608349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.608150 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 02:41:18.614800 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.614777 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bcc8b9b77-htx9w"] Jul 10 02:41:18.633685 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.633638 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:18.752276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752184 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dxx5\" (UniqueName: \"kubernetes.io/projected/7f5d8651-57fe-4157-b6c0-5f051e302eb2-kube-api-access-8dxx5\") pod \"monitoring-plugin-5f68df7f58-fclkz\" (UID: \"7f5d8651-57fe-4157-b6c0-5f051e302eb2\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.752276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752235 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-oauth-config\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.752504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752307 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-serving-cert\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.752504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752357 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-trusted-ca-bundle\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.752504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752388 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8br2l\" (UniqueName: \"kubernetes.io/projected/5011e4a6-924d-4496-a46f-0e2efd82e667-kube-api-access-8br2l\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.752504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752409 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f5d8651-57fe-4157-b6c0-5f051e302eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-fclkz\" (UID: \"7f5d8651-57fe-4157-b6c0-5f051e302eb2\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.752740 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-oauth-serving-cert\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.752740 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752565 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-console-config\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.752740 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.752673 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-service-ca\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.853865 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.853825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-service-ca\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854050 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.853902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dxx5\" (UniqueName: \"kubernetes.io/projected/7f5d8651-57fe-4157-b6c0-5f051e302eb2-kube-api-access-8dxx5\") pod \"monitoring-plugin-5f68df7f58-fclkz\" (UID: \"7f5d8651-57fe-4157-b6c0-5f051e302eb2\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.854050 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.853934 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-oauth-config\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854050 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.853960 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-serving-cert\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854222 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.854196 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-trusted-ca-bundle\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854281 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.854259 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8br2l\" (UniqueName: \"kubernetes.io/projected/5011e4a6-924d-4496-a46f-0e2efd82e667-kube-api-access-8br2l\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854342 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.854292 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f5d8651-57fe-4157-b6c0-5f051e302eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-fclkz\" (UID: \"7f5d8651-57fe-4157-b6c0-5f051e302eb2\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.854394 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.854348 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-oauth-serving-cert\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854394 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.854385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-console-config\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.854606 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.854562 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-service-ca\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.855068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.855018 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-trusted-ca-bundle\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.855170 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.855104 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-console-config\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.855224 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.855201 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-oauth-serving-cert\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.857282 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.857027 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-oauth-config\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.857282 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.857188 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-serving-cert\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.857441 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.857419 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f5d8651-57fe-4157-b6c0-5f051e302eb2-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-fclkz\" (UID: \"7f5d8651-57fe-4157-b6c0-5f051e302eb2\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.861892 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.861871 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dxx5\" (UniqueName: \"kubernetes.io/projected/7f5d8651-57fe-4157-b6c0-5f051e302eb2-kube-api-access-8dxx5\") pod \"monitoring-plugin-5f68df7f58-fclkz\" (UID: \"7f5d8651-57fe-4157-b6c0-5f051e302eb2\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.861999 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.861868 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8br2l\" (UniqueName: \"kubernetes.io/projected/5011e4a6-924d-4496-a46f-0e2efd82e667-kube-api-access-8br2l\") pod \"console-5bcc8b9b77-htx9w\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:18.887476 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.887446 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:18.889143 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.889113 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:18.911723 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:18.911694 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:19.312066 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:19.312027 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:19.413687 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:19.413643 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:19.471921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:19.471870 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:19.980173 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:19.980129 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:21.266098 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:21.266065 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:22.119886 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:22.119848 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:22.220511 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:22.220475 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:22.705432 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:22.705358 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:22.764981 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:22.764943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:24.439284 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.437796 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz"] Jul 10 02:41:24.467345 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.467311 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-878d8cd45-7jsqw"] Jul 10 02:41:24.470026 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:24.469992 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84511177_2df9_477f_9cc7_2420c1500205.slice/crio-3d53549a9b8246194477fb28fa59ee03418d1995d747119462d059a9ac2efefd WatchSource:0}: Error finding container 3d53549a9b8246194477fb28fa59ee03418d1995d747119462d059a9ac2efefd: Status 404 returned error can't find the container with id 3d53549a9b8246194477fb28fa59ee03418d1995d747119462d059a9ac2efefd Jul 10 02:41:24.482390 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.482363 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bcc8b9b77-htx9w"] Jul 10 02:41:24.485853 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:41:24.485832 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5011e4a6_924d_4496_a46f_0e2efd82e667.slice/crio-f5eaf64dd62c64e392e86bd65e6ee591b792f490dbac8b6afdec27f3e256015d WatchSource:0}: Error finding container f5eaf64dd62c64e392e86bd65e6ee591b792f490dbac8b6afdec27f3e256015d: Status 404 returned error can't find the container with id f5eaf64dd62c64e392e86bd65e6ee591b792f490dbac8b6afdec27f3e256015d Jul 10 02:41:24.552627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.552579 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" event={"ID":"7f5d8651-57fe-4157-b6c0-5f051e302eb2","Type":"ContainerStarted","Data":"fe1ca192fc1f1b4bc0b0b7f2fb1499d5efccf13ea62ca193e9fd2e79f092205c"} Jul 10 02:41:24.555253 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.555189 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" event={"ID":"4e8bcddc-4610-4ab6-b030-eb1fbf49fd92","Type":"ContainerStarted","Data":"c5e091871e2d1a3ae137fc3915fd2f5bbd73ebfff05b6c6d6394ca08c07fbe65"} Jul 10 02:41:24.557592 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.557530 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc8b9b77-htx9w" event={"ID":"5011e4a6-924d-4496-a46f-0e2efd82e667","Type":"ContainerStarted","Data":"195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5"} Jul 10 02:41:24.557592 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.557574 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc8b9b77-htx9w" event={"ID":"5011e4a6-924d-4496-a46f-0e2efd82e667","Type":"ContainerStarted","Data":"f5eaf64dd62c64e392e86bd65e6ee591b792f490dbac8b6afdec27f3e256015d"} Jul 10 02:41:24.559416 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.559392 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-457qp" event={"ID":"5f80bd45-9108-40f6-b232-82b2ad2e4b03","Type":"ContainerStarted","Data":"20be49afc5501694b0bff2cb4816eaa1643e40f69aaa3febd79b0cc6e6adaf0f"} Jul 10 02:41:24.560044 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.560031 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:24.561536 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.561512 2585 patch_prober.go:28] interesting pod/downloads-68c9cbb497-457qp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.17:8080/\": dial tcp 10.133.0.17:8080: connect: connection refused" start-of-body= Jul 10 02:41:24.561700 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.561673 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-457qp" podUID="5f80bd45-9108-40f6-b232-82b2ad2e4b03" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.17:8080/\": dial tcp 10.133.0.17:8080: connect: connection refused" Jul 10 02:41:24.562672 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.562599 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qlsbc" event={"ID":"b6059534-5d75-44fe-8288-fb02f5ec5e88","Type":"ContainerStarted","Data":"fc4bb67e53d812e76244518712c6a929479bd653b4a210b7568bdf8bf0cda39f"} Jul 10 02:41:24.563930 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.563897 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" event={"ID":"84511177-2df9-477f-9cc7-2420c1500205","Type":"ContainerStarted","Data":"3d53549a9b8246194477fb28fa59ee03418d1995d747119462d059a9ac2efefd"} Jul 10 02:41:24.600545 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.600481 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-d9nwz" podStartSLOduration=3.323224308 podStartE2EDuration="11.600457335s" totalImagesPullingTime="8.277233027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:24.576563164 +0000 UTC m=+91.285371703" watchObservedRunningTime="2026-07-10 02:41:24.600457335 +0000 UTC m=+91.309265871" Jul 10 02:41:24.600871 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.600839 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qlsbc" podStartSLOduration=10.450987327 podStartE2EDuration="11.600830998s" totalImagesPullingTime="1.072435475s" totalInitContainerRuntime="77.408196ms" isStatefulPod=true podCreationTimestamp="2026-07-10 02:41:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:24.599873115 +0000 UTC m=+91.308681650" watchObservedRunningTime="2026-07-10 02:41:24.600830998 +0000 UTC m=+91.309639536" Jul 10 02:41:24.618211 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.618165 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-457qp" podStartSLOduration=1.413893608 podStartE2EDuration="18.618149575s" totalImagesPullingTime="17.204255967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:24.617382843 +0000 UTC m=+91.326191388" watchObservedRunningTime="2026-07-10 02:41:24.618149575 +0000 UTC m=+91.326958109" Jul 10 02:41:24.644276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:24.644218 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bcc8b9b77-htx9w" podStartSLOduration=6.644197142 podStartE2EDuration="6.644197142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:24.63593791 +0000 UTC m=+91.344746447" watchObservedRunningTime="2026-07-10 02:41:24.644197142 +0000 UTC m=+91.353005677" Jul 10 02:41:25.523276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:25.523233 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:25.580976 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:25.580708 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-457qp" Jul 10 02:41:25.603834 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:25.603750 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:26.519402 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:26.519362 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:26.619138 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:26.619106 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:27.582154 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:27.582072 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" event={"ID":"84511177-2df9-477f-9cc7-2420c1500205","Type":"ContainerStarted","Data":"270adfa5d395ca2ad095ba3096f15e4edfd480cf74eeba94226552d9002027cb"} Jul 10 02:41:27.584021 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:27.583989 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" event={"ID":"7f5d8651-57fe-4157-b6c0-5f051e302eb2","Type":"ContainerStarted","Data":"3e2faf2b77362f9694a97ff9130cb142f13a00059555791cc76295caa39ec59b"} Jul 10 02:41:27.584347 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:27.584324 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:27.589743 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:27.589720 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" Jul 10 02:41:27.600153 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:27.600098 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" podStartSLOduration=6.782014024 podStartE2EDuration="9.600080122s" totalImagesPullingTime="2.818066098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:27.600030708 +0000 UTC m=+94.308839244" watchObservedRunningTime="2026-07-10 02:41:27.600080122 +0000 UTC m=+94.308888660" Jul 10 02:41:27.618078 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:27.618016 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-fclkz" podStartSLOduration=6.765578955 podStartE2EDuration="9.617996752s" totalImagesPullingTime="2.852417797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:27.615510198 +0000 UTC m=+94.324318733" watchObservedRunningTime="2026-07-10 02:41:27.617996752 +0000 UTC m=+94.326805306" Jul 10 02:41:28.317686 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.317591 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:28.392931 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.392897 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:28.477620 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.477585 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d5499fc78-n2xpq" Jul 10 02:41:28.484905 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.484874 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:28.615252 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.615159 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:28.720324 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.720265 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:28.912858 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.912770 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:28.912858 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.912847 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:28.918612 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:28.918586 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:29.597248 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:29.597219 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:41:30.117423 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:30.117386 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:30.218881 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:30.218843 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:30.546889 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:30.546854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:30.647486 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:30.647455 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:31.178640 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:31.178594 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:31.249180 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:31.249122 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:32.440334 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:32.440298 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5wkwz" Jul 10 02:41:32.802063 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:32.801978 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:32.903112 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:32.903075 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:33.004136 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.004090 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:33.490072 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.490014 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" podUID="d542f2a5-c5fe-4e93-b2e7-3209085724c3" containerName="registry" containerID="cri-o://4adbb2de93c875d076e64acd847a551e2ee2d206d957b872c80d36ac8195c966" gracePeriod=30 Jul 10 02:41:33.612210 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.612184 2585 generic.go:356] "Generic (PLEG): container finished" podID="d542f2a5-c5fe-4e93-b2e7-3209085724c3" containerID="4adbb2de93c875d076e64acd847a551e2ee2d206d957b872c80d36ac8195c966" exitCode=0 Jul 10 02:41:33.612339 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.612253 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" event={"ID":"d542f2a5-c5fe-4e93-b2e7-3209085724c3","Type":"ContainerDied","Data":"4adbb2de93c875d076e64acd847a551e2ee2d206d957b872c80d36ac8195c966"} Jul 10 02:41:33.761460 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.761378 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:33.797401 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797369 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-certificates\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797401 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797408 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-installation-pull-secrets\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797437 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d542f2a5-c5fe-4e93-b2e7-3209085724c3-ca-trust-extracted\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797462 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-bound-sa-token\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797500 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2l79\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-kube-api-access-d2l79\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797541 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-trusted-ca\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797584 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-image-registry-private-configuration\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797611 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") pod \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\" (UID: \"d542f2a5-c5fe-4e93-b2e7-3209085724c3\") " Jul 10 02:41:33.797963 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.797920 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:41:33.798498 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.798449 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:41:33.800641 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.800614 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:41:33.800641 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.800616 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-kube-api-access-d2l79" (OuterVolumeSpecName: "kube-api-access-d2l79") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "kube-api-access-d2l79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:41:33.800641 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.800608 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:41:33.800950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.800724 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:41:33.800950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.800885 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:41:33.806606 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.806573 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d542f2a5-c5fe-4e93-b2e7-3209085724c3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d542f2a5-c5fe-4e93-b2e7-3209085724c3" (UID: "d542f2a5-c5fe-4e93-b2e7-3209085724c3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:41:33.898501 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898464 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-trusted-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898501 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898501 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-image-registry-private-configuration\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898501 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898511 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898752 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898522 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d542f2a5-c5fe-4e93-b2e7-3209085724c3-registry-certificates\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898752 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898531 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d542f2a5-c5fe-4e93-b2e7-3209085724c3-installation-pull-secrets\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898752 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898540 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d542f2a5-c5fe-4e93-b2e7-3209085724c3-ca-trust-extracted\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898752 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898548 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-bound-sa-token\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:33.898752 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:33.898555 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2l79\" (UniqueName: \"kubernetes.io/projected/d542f2a5-c5fe-4e93-b2e7-3209085724c3-kube-api-access-d2l79\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:34.618403 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:34.618372 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" Jul 10 02:41:34.618921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:34.618375 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-547685df6b-b8cpb" event={"ID":"d542f2a5-c5fe-4e93-b2e7-3209085724c3","Type":"ContainerDied","Data":"2e2b3fe85faa9ebdb4f85770d9478bbcbdc5f77abacfc783767f93e3b8a9d090"} Jul 10 02:41:34.618921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:34.618508 2585 scope.go:117] "RemoveContainer" containerID="4adbb2de93c875d076e64acd847a551e2ee2d206d957b872c80d36ac8195c966" Jul 10 02:41:34.642323 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:34.642294 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-547685df6b-b8cpb"] Jul 10 02:41:34.646166 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:34.646141 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-547685df6b-b8cpb"] Jul 10 02:41:36.110808 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:36.110775 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d542f2a5-c5fe-4e93-b2e7-3209085724c3" path="/var/lib/kubelet/pods/d542f2a5-c5fe-4e93-b2e7-3209085724c3/volumes" Jul 10 02:41:37.156015 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:37.155976 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:37.256862 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:37.256826 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:38.462011 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:38.461978 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:38.462428 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:38.462039 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:38.947887 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:38.947854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:39.048168 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:39.048128 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:40.253091 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:40.253060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:40.328618 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:40.328584 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:43.607069 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.607000 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b944799f-2n76z" podUID="bc18857c-a776-40a6-a883-780c418b127a" containerName="console" containerID="cri-o://b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464" gracePeriod=15 Jul 10 02:41:43.885173 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.885141 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:43.980408 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980372 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-oauth-config\") pod \"bc18857c-a776-40a6-a883-780c418b127a\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " Jul 10 02:41:43.980408 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980412 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-service-ca\") pod \"bc18857c-a776-40a6-a883-780c418b127a\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " Jul 10 02:41:43.980713 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980443 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-serving-cert\") pod \"bc18857c-a776-40a6-a883-780c418b127a\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " Jul 10 02:41:43.980713 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980474 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-console-config\") pod \"bc18857c-a776-40a6-a883-780c418b127a\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " Jul 10 02:41:43.980713 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980678 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8mbq\" (UniqueName: \"kubernetes.io/projected/bc18857c-a776-40a6-a883-780c418b127a-kube-api-access-w8mbq\") pod \"bc18857c-a776-40a6-a883-780c418b127a\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " Jul 10 02:41:43.980878 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980780 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-oauth-serving-cert\") pod \"bc18857c-a776-40a6-a883-780c418b127a\" (UID: \"bc18857c-a776-40a6-a883-780c418b127a\") " Jul 10 02:41:43.980944 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980900 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-console-config" (OuterVolumeSpecName: "console-config") pod "bc18857c-a776-40a6-a883-780c418b127a" (UID: "bc18857c-a776-40a6-a883-780c418b127a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:41:43.980944 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.980907 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-service-ca" (OuterVolumeSpecName: "service-ca") pod "bc18857c-a776-40a6-a883-780c418b127a" (UID: "bc18857c-a776-40a6-a883-780c418b127a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:41:43.981171 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.981139 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bc18857c-a776-40a6-a883-780c418b127a" (UID: "bc18857c-a776-40a6-a883-780c418b127a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:41:43.981171 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.981156 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:43.981367 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.981189 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-console-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:43.982938 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.982915 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bc18857c-a776-40a6-a883-780c418b127a" (UID: "bc18857c-a776-40a6-a883-780c418b127a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:41:43.983044 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.982982 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bc18857c-a776-40a6-a883-780c418b127a" (UID: "bc18857c-a776-40a6-a883-780c418b127a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:41:43.983044 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:43.983003 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc18857c-a776-40a6-a883-780c418b127a-kube-api-access-w8mbq" (OuterVolumeSpecName: "kube-api-access-w8mbq") pod "bc18857c-a776-40a6-a883-780c418b127a" (UID: "bc18857c-a776-40a6-a883-780c418b127a"). InnerVolumeSpecName "kube-api-access-w8mbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:41:44.082118 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.082076 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-oauth-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:44.082118 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.082103 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc18857c-a776-40a6-a883-780c418b127a-console-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:44.082118 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.082113 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8mbq\" (UniqueName: \"kubernetes.io/projected/bc18857c-a776-40a6-a883-780c418b127a-kube-api-access-w8mbq\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:44.082118 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.082122 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc18857c-a776-40a6-a883-780c418b127a-oauth-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:41:44.659836 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.659799 2585 generic.go:356] "Generic (PLEG): container finished" podID="bc18857c-a776-40a6-a883-780c418b127a" containerID="b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464" exitCode=0 Jul 10 02:41:44.660363 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.659868 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b944799f-2n76z" event={"ID":"bc18857c-a776-40a6-a883-780c418b127a","Type":"ContainerDied","Data":"b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464"} Jul 10 02:41:44.660363 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.660166 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b944799f-2n76z" event={"ID":"bc18857c-a776-40a6-a883-780c418b127a","Type":"ContainerDied","Data":"32ceebb4ea2980942c7a1ab8df8d94daae30ebfff02d26785a74582880e1463d"} Jul 10 02:41:44.660363 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.660249 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b944799f-2n76z" Jul 10 02:41:44.660525 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.660443 2585 scope.go:117] "RemoveContainer" containerID="b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464" Jul 10 02:41:44.671862 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.671830 2585 scope.go:117] "RemoveContainer" containerID="b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464" Jul 10 02:41:44.672161 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:41:44.672140 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464\": container with ID starting with b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464 not found: ID does not exist" containerID="b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464" Jul 10 02:41:44.672200 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.672169 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464"} err="failed to get container status \"b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464\": rpc error: code = NotFound desc = could not find container \"b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464\": container with ID starting with b621c0a4874f0daf363d8479c540bc106e7aa10a2c3a8866788f8a1377217464 not found: ID does not exist" Jul 10 02:41:44.679715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.679679 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b944799f-2n76z"] Jul 10 02:41:44.685509 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:44.685489 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b944799f-2n76z"] Jul 10 02:41:46.111504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:46.111473 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc18857c-a776-40a6-a883-780c418b127a" path="/var/lib/kubelet/pods/bc18857c-a776-40a6-a883-780c418b127a/volumes" Jul 10 02:41:50.684540 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:50.684505 2585 generic.go:356] "Generic (PLEG): container finished" podID="737f8d61-c6e6-4780-95dd-49507bf10d66" containerID="6f68af5820389127916cc11f9fabba5667e245ee79a36c0f8276f187a1829adf" exitCode=0 Jul 10 02:41:50.684900 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:50.684570 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" event={"ID":"737f8d61-c6e6-4780-95dd-49507bf10d66","Type":"ContainerDied","Data":"6f68af5820389127916cc11f9fabba5667e245ee79a36c0f8276f187a1829adf"} Jul 10 02:41:50.684965 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:50.684950 2585 scope.go:117] "RemoveContainer" containerID="6f68af5820389127916cc11f9fabba5667e245ee79a36c0f8276f187a1829adf" Jul 10 02:41:51.690262 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:51.690224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ws8vh" event={"ID":"737f8d61-c6e6-4780-95dd-49507bf10d66","Type":"ContainerStarted","Data":"b21fd44e9147fec1c78d39b8e21166d64fbf8bacdf7b48b395048fae796304d5"} Jul 10 02:41:53.943614 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:53.943581 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:54.005974 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:54.005889 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:41:58.467543 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:58.467509 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:41:58.471349 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:41:58.471320 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-878d8cd45-7jsqw" Jul 10 02:42:05.744165 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:05.744134 2585 generic.go:356] "Generic (PLEG): container finished" podID="b1972ee0-9c38-4c7d-be80-4a6cebda905f" containerID="24e8290e62948ae4f6860c4d7e7a76c702452bba2df8967db5d0f9bc2a1fb3d9" exitCode=0 Jul 10 02:42:05.744560 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:05.744178 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" event={"ID":"b1972ee0-9c38-4c7d-be80-4a6cebda905f","Type":"ContainerDied","Data":"24e8290e62948ae4f6860c4d7e7a76c702452bba2df8967db5d0f9bc2a1fb3d9"} Jul 10 02:42:05.744560 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:05.744511 2585 scope.go:117] "RemoveContainer" containerID="24e8290e62948ae4f6860c4d7e7a76c702452bba2df8967db5d0f9bc2a1fb3d9" Jul 10 02:42:06.399055 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:06.399020 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:06.463082 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:06.463039 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:06.749693 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:06.749642 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rp59d" event={"ID":"b1972ee0-9c38-4c7d-be80-4a6cebda905f","Type":"ContainerStarted","Data":"50458d67aa275d2088f1be9a5fc321fd52fc0fab04cff7041d0277617970b882"} Jul 10 02:42:33.046495 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:33.046455 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:33.497048 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:33.497014 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:38.170576 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170541 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-bbd4bdcc5-28grk"] Jul 10 02:42:38.171032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170822 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d542f2a5-c5fe-4e93-b2e7-3209085724c3" containerName="registry" Jul 10 02:42:38.171032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170834 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d542f2a5-c5fe-4e93-b2e7-3209085724c3" containerName="registry" Jul 10 02:42:38.171032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170842 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc18857c-a776-40a6-a883-780c418b127a" containerName="console" Jul 10 02:42:38.171032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170848 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc18857c-a776-40a6-a883-780c418b127a" containerName="console" Jul 10 02:42:38.171032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170921 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d542f2a5-c5fe-4e93-b2e7-3209085724c3" containerName="registry" Jul 10 02:42:38.171032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.170930 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc18857c-a776-40a6-a883-780c418b127a" containerName="console" Jul 10 02:42:38.173431 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.173411 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.177171 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.177144 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 02:42:38.177278 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.177261 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g9lx4\"" Jul 10 02:42:38.177447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.177433 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 02:42:38.177564 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.177546 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 02:42:38.177686 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.177672 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 02:42:38.177884 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.177870 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 02:42:38.183897 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.183878 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 02:42:38.196892 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.196871 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bbd4bdcc5-28grk"] Jul 10 02:42:38.199620 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199599 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-federate-client-tls\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199799 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199696 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkww5\" (UniqueName: \"kubernetes.io/projected/f727ff78-9a77-4860-a4f9-779b745d6273-kube-api-access-wkww5\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199799 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199747 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-serving-certs-ca-bundle\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199799 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199791 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-metrics-client-ca\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-secret-telemeter-client\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199869 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.199950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.199933 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-telemeter-client-tls\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301037 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301005 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-telemeter-client-tls\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301233 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301052 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-federate-client-tls\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301233 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301074 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkww5\" (UniqueName: \"kubernetes.io/projected/f727ff78-9a77-4860-a4f9-779b745d6273-kube-api-access-wkww5\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301233 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301179 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-serving-certs-ca-bundle\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301233 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301212 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-metrics-client-ca\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301233 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301235 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-secret-telemeter-client\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301270 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301328 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.301995 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301963 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-serving-certs-ca-bundle\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.302118 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.301973 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-metrics-client-ca\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.302218 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.302195 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f727ff78-9a77-4860-a4f9-779b745d6273-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.303605 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.303580 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-federate-client-tls\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.303746 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.303631 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-secret-telemeter-client\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.303986 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.303966 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-telemeter-client-tls\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.304048 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.304013 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f727ff78-9a77-4860-a4f9-779b745d6273-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.310134 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.310117 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkww5\" (UniqueName: \"kubernetes.io/projected/f727ff78-9a77-4860-a4f9-779b745d6273-kube-api-access-wkww5\") pod \"telemeter-client-bbd4bdcc5-28grk\" (UID: \"f727ff78-9a77-4860-a4f9-779b745d6273\") " pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.325321 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.325297 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:38.483373 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.483337 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" Jul 10 02:42:38.619242 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.619220 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bbd4bdcc5-28grk"] Jul 10 02:42:38.621476 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:42:38.621453 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf727ff78_9a77_4860_a4f9_779b745d6273.slice/crio-9ba534f03b932231dfdb29a5d6230606c861b945fb28ec032d8b4b6c1db3b288 WatchSource:0}: Error finding container 9ba534f03b932231dfdb29a5d6230606c861b945fb28ec032d8b4b6c1db3b288: Status 404 returned error can't find the container with id 9ba534f03b932231dfdb29a5d6230606c861b945fb28ec032d8b4b6c1db3b288 Jul 10 02:42:38.870241 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:38.870149 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" event={"ID":"f727ff78-9a77-4860-a4f9-779b745d6273","Type":"ContainerStarted","Data":"9ba534f03b932231dfdb29a5d6230606c861b945fb28ec032d8b4b6c1db3b288"} Jul 10 02:42:40.878891 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:40.878857 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" event={"ID":"f727ff78-9a77-4860-a4f9-779b745d6273","Type":"ContainerStarted","Data":"fae562c5756081cbbea740afb4852aaca01bf3d7ef92c93b74c30b7363da4a68"} Jul 10 02:42:41.889158 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:41.889122 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" event={"ID":"f727ff78-9a77-4860-a4f9-779b745d6273","Type":"ContainerStarted","Data":"067d5f3dd911517d89961178a0fe8ceb6195856d5ec4c58a7641636a561d9068"} Jul 10 02:42:41.889516 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:41.889208 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" event={"ID":"f727ff78-9a77-4860-a4f9-779b745d6273","Type":"ContainerStarted","Data":"59a8027b391cc3a61034d3211e2d45ee4abd6c4cce611813b74be9dc64e78e1d"} Jul 10 02:42:41.921159 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:41.921103 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-bbd4bdcc5-28grk" podStartSLOduration=0.884618487 podStartE2EDuration="3.921089508s" totalImagesPullingTime="3.036471021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:41.920916281 +0000 UTC m=+168.629724819" watchObservedRunningTime="2026-07-10 02:42:41.921089508 +0000 UTC m=+168.629898042" Jul 10 02:42:42.922600 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:42.922568 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66d5bd468f-xzjzt"] Jul 10 02:42:42.924946 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:42.924925 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:42.937341 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:42.937314 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d5bd468f-xzjzt"] Jul 10 02:42:42.979737 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:42.979679 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:43.044938 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.044904 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnw99\" (UniqueName: \"kubernetes.io/projected/a1aa1944-9957-4375-a56f-5304231e1be4-kube-api-access-rnw99\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.044938 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.044941 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-oauth-serving-cert\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.045184 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.044975 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-console-config\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.045184 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.045047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-service-ca\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.045184 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.045100 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-trusted-ca-bundle\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.045184 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.045130 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-oauth-config\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.045184 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.045163 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-serving-cert\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.082198 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.082171 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:43.146449 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146410 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnw99\" (UniqueName: \"kubernetes.io/projected/a1aa1944-9957-4375-a56f-5304231e1be4-kube-api-access-rnw99\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.146449 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146452 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-oauth-serving-cert\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.146753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146469 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-console-config\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.146753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146487 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-service-ca\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.146753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146514 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-trusted-ca-bundle\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.146753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146545 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-oauth-config\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.146753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.146593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-serving-cert\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.147333 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.147307 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-service-ca\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.147445 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.147334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-console-config\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.147445 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.147358 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-oauth-serving-cert\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.147515 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.147463 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-trusted-ca-bundle\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.149155 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.149128 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-serving-cert\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.149231 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.149186 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-oauth-config\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.155680 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.155659 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnw99\" (UniqueName: \"kubernetes.io/projected/a1aa1944-9957-4375-a56f-5304231e1be4-kube-api-access-rnw99\") pod \"console-66d5bd468f-xzjzt\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.180979 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.180939 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:43.235321 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.235292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:43.371924 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.371891 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d5bd468f-xzjzt"] Jul 10 02:42:43.374597 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:42:43.374569 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1aa1944_9957_4375_a56f_5304231e1be4.slice/crio-c5aedd9f58541da1d6dcb408b396b8a0214053a23cfcecf9a60679f14eb7d1f7 WatchSource:0}: Error finding container c5aedd9f58541da1d6dcb408b396b8a0214053a23cfcecf9a60679f14eb7d1f7: Status 404 returned error can't find the container with id c5aedd9f58541da1d6dcb408b396b8a0214053a23cfcecf9a60679f14eb7d1f7 Jul 10 02:42:43.715156 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.715064 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:43.817169 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.817134 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:43.897993 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.897957 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d5bd468f-xzjzt" event={"ID":"a1aa1944-9957-4375-a56f-5304231e1be4","Type":"ContainerStarted","Data":"3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430"} Jul 10 02:42:43.898176 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.898008 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d5bd468f-xzjzt" event={"ID":"a1aa1944-9957-4375-a56f-5304231e1be4","Type":"ContainerStarted","Data":"c5aedd9f58541da1d6dcb408b396b8a0214053a23cfcecf9a60679f14eb7d1f7"} Jul 10 02:42:43.918367 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:43.918311 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66d5bd468f-xzjzt" podStartSLOduration=1.918295176 podStartE2EDuration="1.918295176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:43.91679707 +0000 UTC m=+170.625605607" watchObservedRunningTime="2026-07-10 02:42:43.918295176 +0000 UTC m=+170.627103711" Jul 10 02:42:53.235632 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:53.235552 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:53.235632 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:53.235613 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:53.240525 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:53.240502 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:53.939978 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:53.939949 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:42:53.989028 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:53.989001 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bcc8b9b77-htx9w"] Jul 10 02:42:54.100834 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:54.100804 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:54.202785 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:54.202701 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:42:54.266274 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:42:54.266246 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:19.015108 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.015044 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bcc8b9b77-htx9w" podUID="5011e4a6-924d-4496-a46f-0e2efd82e667" containerName="console" containerID="cri-o://195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5" gracePeriod=15 Jul 10 02:43:19.277775 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.277719 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:43:19.363235 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363200 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-oauth-serving-cert\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363235 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363237 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-trusted-ca-bundle\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363266 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-serving-cert\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363293 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-oauth-config\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363318 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-service-ca\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363336 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8br2l\" (UniqueName: \"kubernetes.io/projected/5011e4a6-924d-4496-a46f-0e2efd82e667-kube-api-access-8br2l\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363351 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-console-config\") pod \"5011e4a6-924d-4496-a46f-0e2efd82e667\" (UID: \"5011e4a6-924d-4496-a46f-0e2efd82e667\") " Jul 10 02:43:19.363849 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363790 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:43:19.363969 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363932 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-console-config" (OuterVolumeSpecName: "console-config") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:43:19.363969 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363953 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-service-ca" (OuterVolumeSpecName: "service-ca") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:43:19.363969 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.363946 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:43:19.365495 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.365473 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:43:19.365576 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.365520 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:43:19.365576 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.365531 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5011e4a6-924d-4496-a46f-0e2efd82e667-kube-api-access-8br2l" (OuterVolumeSpecName: "kube-api-access-8br2l") pod "5011e4a6-924d-4496-a46f-0e2efd82e667" (UID: "5011e4a6-924d-4496-a46f-0e2efd82e667"). InnerVolumeSpecName "kube-api-access-8br2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:43:19.464910 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464875 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-oauth-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:19.464910 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464901 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-trusted-ca-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:19.464910 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464910 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:19.464910 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464919 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5011e4a6-924d-4496-a46f-0e2efd82e667-console-oauth-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:19.465167 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464929 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:19.465167 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464937 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8br2l\" (UniqueName: \"kubernetes.io/projected/5011e4a6-924d-4496-a46f-0e2efd82e667-kube-api-access-8br2l\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:19.465167 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:19.464982 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5011e4a6-924d-4496-a46f-0e2efd82e667-console-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:43:20.042259 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.042225 2585 generic.go:356] "Generic (PLEG): container finished" podID="5011e4a6-924d-4496-a46f-0e2efd82e667" containerID="195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5" exitCode=0 Jul 10 02:43:20.042667 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.042272 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcc8b9b77-htx9w" Jul 10 02:43:20.042667 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.042300 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc8b9b77-htx9w" event={"ID":"5011e4a6-924d-4496-a46f-0e2efd82e667","Type":"ContainerDied","Data":"195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5"} Jul 10 02:43:20.042667 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.042346 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcc8b9b77-htx9w" event={"ID":"5011e4a6-924d-4496-a46f-0e2efd82e667","Type":"ContainerDied","Data":"f5eaf64dd62c64e392e86bd65e6ee591b792f490dbac8b6afdec27f3e256015d"} Jul 10 02:43:20.042667 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.042364 2585 scope.go:117] "RemoveContainer" containerID="195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5" Jul 10 02:43:20.050829 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.050813 2585 scope.go:117] "RemoveContainer" containerID="195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5" Jul 10 02:43:20.051077 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:43:20.051060 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5\": container with ID starting with 195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5 not found: ID does not exist" containerID="195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5" Jul 10 02:43:20.051157 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.051087 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5"} err="failed to get container status \"195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5\": rpc error: code = NotFound desc = could not find container \"195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5\": container with ID starting with 195274dd4d8d7c156c30fff998db42da85e9eef79464f6a7a73bc4b4d19ad7f5 not found: ID does not exist" Jul 10 02:43:20.064243 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.064218 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bcc8b9b77-htx9w"] Jul 10 02:43:20.068368 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.068344 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bcc8b9b77-htx9w"] Jul 10 02:43:20.111131 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:20.111105 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5011e4a6-924d-4496-a46f-0e2efd82e667" path="/var/lib/kubelet/pods/5011e4a6-924d-4496-a46f-0e2efd82e667/volumes" Jul 10 02:43:23.242132 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:23.242093 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:32.838591 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:32.838557 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:48.329995 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:48.329962 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:48.431040 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:48.431002 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:52.456324 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:52.456285 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:55.352749 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.352706 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688c6dc994-z8m27"] Jul 10 02:43:55.353193 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.353108 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5011e4a6-924d-4496-a46f-0e2efd82e667" containerName="console" Jul 10 02:43:55.353193 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.353126 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5011e4a6-924d-4496-a46f-0e2efd82e667" containerName="console" Jul 10 02:43:55.353309 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.353207 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5011e4a6-924d-4496-a46f-0e2efd82e667" containerName="console" Jul 10 02:43:55.357384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.357361 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.367300 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.367271 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688c6dc994-z8m27"] Jul 10 02:43:55.410279 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.410243 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:55.458665 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458615 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-serving-cert\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.458835 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458675 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-oauth-serving-cert\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.458835 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458768 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-trusted-ca-bundle\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.458926 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458846 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc429\" (UniqueName: \"kubernetes.io/projected/88f49595-8ba2-46ff-ba46-39d304741499-kube-api-access-fc429\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.458926 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458874 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-service-ca\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.459005 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458931 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-oauth-config\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.459005 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.458963 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-console-config\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.472912 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.472880 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:43:55.560355 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560315 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc429\" (UniqueName: \"kubernetes.io/projected/88f49595-8ba2-46ff-ba46-39d304741499-kube-api-access-fc429\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.560355 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560353 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-service-ca\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.560579 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560473 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-oauth-config\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.560579 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560513 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-console-config\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.560579 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-serving-cert\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.560748 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560580 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-oauth-serving-cert\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.560819 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.560792 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-trusted-ca-bundle\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.561222 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.561196 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-service-ca\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.561338 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.561235 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-oauth-serving-cert\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.561338 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.561293 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-console-config\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.561493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.561474 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-trusted-ca-bundle\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.562966 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.562936 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-oauth-config\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.563109 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.563087 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-serving-cert\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.570014 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.569991 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc429\" (UniqueName: \"kubernetes.io/projected/88f49595-8ba2-46ff-ba46-39d304741499-kube-api-access-fc429\") pod \"console-688c6dc994-z8m27\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.667455 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.667364 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:43:55.797445 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:55.797423 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688c6dc994-z8m27"] Jul 10 02:43:55.799618 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:43:55.799584 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f49595_8ba2_46ff_ba46_39d304741499.slice/crio-1ff5589497b7887aa18fd7aba832ea600a8a41b81ecedb4f96995e3e3efc1c5b WatchSource:0}: Error finding container 1ff5589497b7887aa18fd7aba832ea600a8a41b81ecedb4f96995e3e3efc1c5b: Status 404 returned error can't find the container with id 1ff5589497b7887aa18fd7aba832ea600a8a41b81ecedb4f96995e3e3efc1c5b Jul 10 02:43:56.178364 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:56.178330 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c6dc994-z8m27" event={"ID":"88f49595-8ba2-46ff-ba46-39d304741499","Type":"ContainerStarted","Data":"767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835"} Jul 10 02:43:56.178570 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:56.178375 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c6dc994-z8m27" event={"ID":"88f49595-8ba2-46ff-ba46-39d304741499","Type":"ContainerStarted","Data":"1ff5589497b7887aa18fd7aba832ea600a8a41b81ecedb4f96995e3e3efc1c5b"} Jul 10 02:43:56.197930 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:43:56.197861 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688c6dc994-z8m27" podStartSLOduration=1.197840995 podStartE2EDuration="1.197840995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:43:56.196625756 +0000 UTC m=+242.905434307" watchObservedRunningTime="2026-07-10 02:43:56.197840995 +0000 UTC m=+242.906649540" Jul 10 02:44:00.321617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:00.321582 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:00.391037 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:00.391004 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:05.668113 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:05.668070 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:44:05.668517 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:05.668162 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:44:05.672858 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:05.672834 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:44:06.218414 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:06.218380 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:44:06.269412 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:06.269382 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d5bd468f-xzjzt"] Jul 10 02:44:06.384320 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:06.384283 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:06.485728 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:06.485600 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:06.585848 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:06.585804 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:08.948886 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:08.948848 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:09.049754 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:09.049707 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:31.289499 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.289444 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66d5bd468f-xzjzt" podUID="a1aa1944-9957-4375-a56f-5304231e1be4" containerName="console" containerID="cri-o://3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430" gracePeriod=15 Jul 10 02:44:31.526494 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.526464 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:44:31.659075 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.658989 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-serving-cert\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659075 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659042 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-oauth-serving-cert\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659095 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-console-config\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659113 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-oauth-config\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659134 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-service-ca\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659153 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnw99\" (UniqueName: \"kubernetes.io/projected/a1aa1944-9957-4375-a56f-5304231e1be4-kube-api-access-rnw99\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659186 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-trusted-ca-bundle\") pod \"a1aa1944-9957-4375-a56f-5304231e1be4\" (UID: \"a1aa1944-9957-4375-a56f-5304231e1be4\") " Jul 10 02:44:31.659634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659601 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:44:31.659634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659610 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-console-config" (OuterVolumeSpecName: "console-config") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:44:31.659634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659624 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-service-ca" (OuterVolumeSpecName: "service-ca") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:44:31.659874 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.659785 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:44:31.661174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.661155 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:44:31.661293 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.661270 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:44:31.661329 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.661292 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1aa1944-9957-4375-a56f-5304231e1be4-kube-api-access-rnw99" (OuterVolumeSpecName: "kube-api-access-rnw99") pod "a1aa1944-9957-4375-a56f-5304231e1be4" (UID: "a1aa1944-9957-4375-a56f-5304231e1be4"). InnerVolumeSpecName "kube-api-access-rnw99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:44:31.760665 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760616 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-oauth-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:31.760665 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760663 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-console-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:31.760665 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760673 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-oauth-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:31.760894 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760681 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:31.760894 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760689 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnw99\" (UniqueName: \"kubernetes.io/projected/a1aa1944-9957-4375-a56f-5304231e1be4-kube-api-access-rnw99\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:31.760894 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760697 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a1aa1944-9957-4375-a56f-5304231e1be4-trusted-ca-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:31.760894 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:31.760705 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a1aa1944-9957-4375-a56f-5304231e1be4-console-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:44:32.309779 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.309690 2585 generic.go:356] "Generic (PLEG): container finished" podID="a1aa1944-9957-4375-a56f-5304231e1be4" containerID="3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430" exitCode=0 Jul 10 02:44:32.309779 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.309746 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d5bd468f-xzjzt" Jul 10 02:44:32.309779 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.309766 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d5bd468f-xzjzt" event={"ID":"a1aa1944-9957-4375-a56f-5304231e1be4","Type":"ContainerDied","Data":"3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430"} Jul 10 02:44:32.310270 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.309840 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d5bd468f-xzjzt" event={"ID":"a1aa1944-9957-4375-a56f-5304231e1be4","Type":"ContainerDied","Data":"c5aedd9f58541da1d6dcb408b396b8a0214053a23cfcecf9a60679f14eb7d1f7"} Jul 10 02:44:32.310270 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.309860 2585 scope.go:117] "RemoveContainer" containerID="3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430" Jul 10 02:44:32.317816 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.317799 2585 scope.go:117] "RemoveContainer" containerID="3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430" Jul 10 02:44:32.318070 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:44:32.318050 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430\": container with ID starting with 3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430 not found: ID does not exist" containerID="3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430" Jul 10 02:44:32.318128 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.318080 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430"} err="failed to get container status \"3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430\": rpc error: code = NotFound desc = could not find container \"3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430\": container with ID starting with 3b366e2cafb0987a322448c4d2e057c2fab33c40b1e4ebc55e671492f4645430 not found: ID does not exist" Jul 10 02:44:32.327290 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.327260 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66d5bd468f-xzjzt"] Jul 10 02:44:32.331033 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:32.331012 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66d5bd468f-xzjzt"] Jul 10 02:44:34.110306 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:34.110276 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1aa1944-9957-4375-a56f-5304231e1be4" path="/var/lib/kubelet/pods/a1aa1944-9957-4375-a56f-5304231e1be4/volumes" Jul 10 02:44:39.621708 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:39.621671 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:39.715815 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:39.715764 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:40.426297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.426264 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l"] Jul 10 02:44:40.426611 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.426598 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1aa1944-9957-4375-a56f-5304231e1be4" containerName="console" Jul 10 02:44:40.426675 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.426612 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1aa1944-9957-4375-a56f-5304231e1be4" containerName="console" Jul 10 02:44:40.426713 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.426682 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1aa1944-9957-4375-a56f-5304231e1be4" containerName="console" Jul 10 02:44:40.430929 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.430912 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.433629 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.433607 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 02:44:40.434821 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.434805 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 02:44:40.434821 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.434812 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m8gzs\"" Jul 10 02:44:40.439217 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.439196 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l"] Jul 10 02:44:40.535949 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.535911 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.536137 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.535956 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.536137 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.536027 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vhcc\" (UniqueName: \"kubernetes.io/projected/2d4350f2-2797-4b9c-81d1-016da470954e-kube-api-access-6vhcc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.637185 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.637136 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.637686 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.637211 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vhcc\" (UniqueName: \"kubernetes.io/projected/2d4350f2-2797-4b9c-81d1-016da470954e-kube-api-access-6vhcc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.637686 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.637298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.637686 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.637527 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.637686 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.637617 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.648906 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.648875 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vhcc\" (UniqueName: \"kubernetes.io/projected/2d4350f2-2797-4b9c-81d1-016da470954e-kube-api-access-6vhcc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.740226 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.740128 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:44:40.874322 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:40.874120 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l"] Jul 10 02:44:40.876616 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:44:40.876580 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4350f2_2797_4b9c_81d1_016da470954e.slice/crio-f016bf9f738e25a561417fc59a9fbfba696d0bfad64cee8f833a257afb36a758 WatchSource:0}: Error finding container f016bf9f738e25a561417fc59a9fbfba696d0bfad64cee8f833a257afb36a758: Status 404 returned error can't find the container with id f016bf9f738e25a561417fc59a9fbfba696d0bfad64cee8f833a257afb36a758 Jul 10 02:44:41.349278 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:41.349232 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" event={"ID":"2d4350f2-2797-4b9c-81d1-016da470954e","Type":"ContainerStarted","Data":"f016bf9f738e25a561417fc59a9fbfba696d0bfad64cee8f833a257afb36a758"} Jul 10 02:44:42.843620 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:42.843588 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:44:49.384044 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:49.383952 2585 generic.go:356] "Generic (PLEG): container finished" podID="2d4350f2-2797-4b9c-81d1-016da470954e" containerID="ba480f99f3c7e31dc09ca176c41a3c0d96f19fb39a61cf2d2fc71227f3153c25" exitCode=0 Jul 10 02:44:49.384044 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:49.384002 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" event={"ID":"2d4350f2-2797-4b9c-81d1-016da470954e","Type":"ContainerDied","Data":"ba480f99f3c7e31dc09ca176c41a3c0d96f19fb39a61cf2d2fc71227f3153c25"} Jul 10 02:44:51.394178 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:51.394139 2585 generic.go:356] "Generic (PLEG): container finished" podID="2d4350f2-2797-4b9c-81d1-016da470954e" containerID="4ca1d52a39251ad04eb440fc030d1c33b0e78074cb7006e6676d913648ed0350" exitCode=0 Jul 10 02:44:51.394545 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:51.394201 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" event={"ID":"2d4350f2-2797-4b9c-81d1-016da470954e","Type":"ContainerDied","Data":"4ca1d52a39251ad04eb440fc030d1c33b0e78074cb7006e6676d913648ed0350"} Jul 10 02:44:53.751009 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:53.750915 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:44:53.751009 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:53.750974 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:44:53.763579 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:53.763557 2585 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 02:44:59.428082 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:59.428050 2585 generic.go:356] "Generic (PLEG): container finished" podID="2d4350f2-2797-4b9c-81d1-016da470954e" containerID="be3e939c8ad69f4c181b2d41f8375658b9d47018880e7130970dbbc842ce37ce" exitCode=0 Jul 10 02:44:59.428474 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:44:59.428139 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" event={"ID":"2d4350f2-2797-4b9c-81d1-016da470954e","Type":"ContainerDied","Data":"be3e939c8ad69f4c181b2d41f8375658b9d47018880e7130970dbbc842ce37ce"} Jul 10 02:45:00.562322 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.562298 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:45:00.711209 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.711106 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-bundle\") pod \"2d4350f2-2797-4b9c-81d1-016da470954e\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " Jul 10 02:45:00.711209 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.711175 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-util\") pod \"2d4350f2-2797-4b9c-81d1-016da470954e\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " Jul 10 02:45:00.711460 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.711225 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vhcc\" (UniqueName: \"kubernetes.io/projected/2d4350f2-2797-4b9c-81d1-016da470954e-kube-api-access-6vhcc\") pod \"2d4350f2-2797-4b9c-81d1-016da470954e\" (UID: \"2d4350f2-2797-4b9c-81d1-016da470954e\") " Jul 10 02:45:00.711786 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.711759 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-bundle" (OuterVolumeSpecName: "bundle") pod "2d4350f2-2797-4b9c-81d1-016da470954e" (UID: "2d4350f2-2797-4b9c-81d1-016da470954e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:45:00.713472 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.713448 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4350f2-2797-4b9c-81d1-016da470954e-kube-api-access-6vhcc" (OuterVolumeSpecName: "kube-api-access-6vhcc") pod "2d4350f2-2797-4b9c-81d1-016da470954e" (UID: "2d4350f2-2797-4b9c-81d1-016da470954e"). InnerVolumeSpecName "kube-api-access-6vhcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:45:00.715463 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.715443 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-util" (OuterVolumeSpecName: "util") pod "2d4350f2-2797-4b9c-81d1-016da470954e" (UID: "2d4350f2-2797-4b9c-81d1-016da470954e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:45:00.812099 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.812066 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vhcc\" (UniqueName: \"kubernetes.io/projected/2d4350f2-2797-4b9c-81d1-016da470954e-kube-api-access-6vhcc\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:00.812099 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.812095 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:00.812099 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:00.812104 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d4350f2-2797-4b9c-81d1-016da470954e-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:01.437528 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:01.437496 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" Jul 10 02:45:01.437528 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:01.437516 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ng79l" event={"ID":"2d4350f2-2797-4b9c-81d1-016da470954e","Type":"ContainerDied","Data":"f016bf9f738e25a561417fc59a9fbfba696d0bfad64cee8f833a257afb36a758"} Jul 10 02:45:01.437783 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:01.437565 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f016bf9f738e25a561417fc59a9fbfba696d0bfad64cee8f833a257afb36a758" Jul 10 02:45:06.491099 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:06.491059 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:08.062775 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.062742 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:08.494869 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.494837 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8"] Jul 10 02:45:08.495136 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495125 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="util" Jul 10 02:45:08.495181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495137 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="util" Jul 10 02:45:08.495181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495152 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="extract" Jul 10 02:45:08.495181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495158 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="extract" Jul 10 02:45:08.495181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495166 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="pull" Jul 10 02:45:08.495181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495171 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="pull" Jul 10 02:45:08.495321 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.495220 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d4350f2-2797-4b9c-81d1-016da470954e" containerName="extract" Jul 10 02:45:08.499280 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.499261 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.502711 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.502685 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mvhlm\"" Jul 10 02:45:08.503269 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.503244 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 02:45:08.503370 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.503248 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:45:08.513532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.513506 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8"] Jul 10 02:45:08.570336 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.570295 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a09e278a-f42e-41bb-a91b-07f4d42c93b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v9jv8\" (UID: \"a09e278a-f42e-41bb-a91b-07f4d42c93b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.570507 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.570406 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gvjk\" (UniqueName: \"kubernetes.io/projected/a09e278a-f42e-41bb-a91b-07f4d42c93b7-kube-api-access-8gvjk\") pod \"cert-manager-operator-controller-manager-55c444bf86-v9jv8\" (UID: \"a09e278a-f42e-41bb-a91b-07f4d42c93b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.670776 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.670738 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gvjk\" (UniqueName: \"kubernetes.io/projected/a09e278a-f42e-41bb-a91b-07f4d42c93b7-kube-api-access-8gvjk\") pod \"cert-manager-operator-controller-manager-55c444bf86-v9jv8\" (UID: \"a09e278a-f42e-41bb-a91b-07f4d42c93b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.670915 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.670792 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a09e278a-f42e-41bb-a91b-07f4d42c93b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v9jv8\" (UID: \"a09e278a-f42e-41bb-a91b-07f4d42c93b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.671234 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.671217 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a09e278a-f42e-41bb-a91b-07f4d42c93b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-v9jv8\" (UID: \"a09e278a-f42e-41bb-a91b-07f4d42c93b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.682541 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.682515 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gvjk\" (UniqueName: \"kubernetes.io/projected/a09e278a-f42e-41bb-a91b-07f4d42c93b7-kube-api-access-8gvjk\") pod \"cert-manager-operator-controller-manager-55c444bf86-v9jv8\" (UID: \"a09e278a-f42e-41bb-a91b-07f4d42c93b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.808345 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.808248 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" Jul 10 02:45:08.939638 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.939616 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8"] Jul 10 02:45:08.942008 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:08.941980 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09e278a_f42e_41bb_a91b_07f4d42c93b7.slice/crio-976269f0c07c72c33c3065456b4b6d72b1d786c791bb6d746b38cc22c0e15554 WatchSource:0}: Error finding container 976269f0c07c72c33c3065456b4b6d72b1d786c791bb6d746b38cc22c0e15554: Status 404 returned error can't find the container with id 976269f0c07c72c33c3065456b4b6d72b1d786c791bb6d746b38cc22c0e15554 Jul 10 02:45:08.947730 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:08.947707 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 02:45:09.470434 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:09.470399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" event={"ID":"a09e278a-f42e-41bb-a91b-07f4d42c93b7","Type":"ContainerStarted","Data":"976269f0c07c72c33c3065456b4b6d72b1d786c791bb6d746b38cc22c0e15554"} Jul 10 02:45:10.110964 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:10.110926 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:10.210050 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:10.209998 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:11.480814 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:11.480781 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" event={"ID":"a09e278a-f42e-41bb-a91b-07f4d42c93b7","Type":"ContainerStarted","Data":"8a2aca7aad1c1c68970bdd792a6b4c14ab2ef7a0db463c3eaff71aa1353e041c"} Jul 10 02:45:11.506683 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:11.506608 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-v9jv8" podStartSLOduration=1.7442134820000001 podStartE2EDuration="3.506590125s" totalImagesPullingTime="1.762376643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:11.504626144 +0000 UTC m=+318.213434679" watchObservedRunningTime="2026-07-10 02:45:11.506590125 +0000 UTC m=+318.215398662" Jul 10 02:45:11.634359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:11.634326 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:11.735143 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:11.735060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:11.948220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:11.948190 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:12.074397 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.074309 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:12.903033 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.902998 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg"] Jul 10 02:45:12.905593 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.905571 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:12.909186 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.909169 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 02:45:12.910296 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.910281 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 02:45:12.910395 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.910298 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m8gzs\"" Jul 10 02:45:12.923629 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:12.923604 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg"] Jul 10 02:45:13.002369 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.002335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.002546 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.002386 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqnl6\" (UniqueName: \"kubernetes.io/projected/d2e36d62-d432-4f0c-a07c-20239d93c70c-kube-api-access-tqnl6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.002546 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.002481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.102864 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.102831 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.103038 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.102881 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.103038 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.102926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqnl6\" (UniqueName: \"kubernetes.io/projected/d2e36d62-d432-4f0c-a07c-20239d93c70c-kube-api-access-tqnl6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.103290 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.103260 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.103392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.103302 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.111877 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.111846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqnl6\" (UniqueName: \"kubernetes.io/projected/d2e36d62-d432-4f0c-a07c-20239d93c70c-kube-api-access-tqnl6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.214186 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.214147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:13.351082 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.351054 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg"] Jul 10 02:45:13.354188 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:13.354162 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e36d62_d432_4f0c_a07c_20239d93c70c.slice/crio-9532b3f0bcacb63542b4a7e9859bf883d1c5acc48701c3ffc2b73d649bfd5f86 WatchSource:0}: Error finding container 9532b3f0bcacb63542b4a7e9859bf883d1c5acc48701c3ffc2b73d649bfd5f86: Status 404 returned error can't find the container with id 9532b3f0bcacb63542b4a7e9859bf883d1c5acc48701c3ffc2b73d649bfd5f86 Jul 10 02:45:13.492094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.492013 2585 generic.go:356] "Generic (PLEG): container finished" podID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerID="43ebc5bb2e45dae6c79cc39c8c514ec51a0674e31f55b5f561ba3e7b3451c0b8" exitCode=0 Jul 10 02:45:13.492228 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.492111 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" event={"ID":"d2e36d62-d432-4f0c-a07c-20239d93c70c","Type":"ContainerDied","Data":"43ebc5bb2e45dae6c79cc39c8c514ec51a0674e31f55b5f561ba3e7b3451c0b8"} Jul 10 02:45:13.492228 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:13.492178 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" event={"ID":"d2e36d62-d432-4f0c-a07c-20239d93c70c","Type":"ContainerStarted","Data":"9532b3f0bcacb63542b4a7e9859bf883d1c5acc48701c3ffc2b73d649bfd5f86"} Jul 10 02:45:14.036096 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:14.036056 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:14.445984 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:14.445952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:15.503223 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.503197 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" event={"ID":"d2e36d62-d432-4f0c-a07c-20239d93c70c","Type":"ContainerStarted","Data":"80f170552e90d2ca333b6dea079965dc2a6ff6af6672c89d92a5258f5f1db1e4"} Jul 10 02:45:15.517797 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.517771 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-48d6h"] Jul 10 02:45:15.519900 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.519883 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.522387 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.522355 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 02:45:15.522387 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.522365 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 02:45:15.522541 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.522419 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-brmtz\"" Jul 10 02:45:15.530637 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.530618 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-48d6h"] Jul 10 02:45:15.622964 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.622931 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86wxz\" (UniqueName: \"kubernetes.io/projected/5a3600b1-9342-45e8-9e9a-9adc1e4c33a3-kube-api-access-86wxz\") pod \"cert-manager-cainjector-559cdd8c87-48d6h\" (UID: \"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.623101 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.622971 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a3600b1-9342-45e8-9e9a-9adc1e4c33a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-48d6h\" (UID: \"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.723344 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.723310 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86wxz\" (UniqueName: \"kubernetes.io/projected/5a3600b1-9342-45e8-9e9a-9adc1e4c33a3-kube-api-access-86wxz\") pod \"cert-manager-cainjector-559cdd8c87-48d6h\" (UID: \"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.723344 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.723347 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a3600b1-9342-45e8-9e9a-9adc1e4c33a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-48d6h\" (UID: \"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.731822 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.731793 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a3600b1-9342-45e8-9e9a-9adc1e4c33a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-48d6h\" (UID: \"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.731962 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.731927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86wxz\" (UniqueName: \"kubernetes.io/projected/5a3600b1-9342-45e8-9e9a-9adc1e4c33a3-kube-api-access-86wxz\") pod \"cert-manager-cainjector-559cdd8c87-48d6h\" (UID: \"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.840960 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.840913 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" Jul 10 02:45:15.982934 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:15.982906 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-48d6h"] Jul 10 02:45:15.984607 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:15.984582 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a3600b1_9342_45e8_9e9a_9adc1e4c33a3.slice/crio-ffe0a1cf82a3f479313d71f9513e9baca836e1583f28494fc25d0ca2eba7d80e WatchSource:0}: Error finding container ffe0a1cf82a3f479313d71f9513e9baca836e1583f28494fc25d0ca2eba7d80e: Status 404 returned error can't find the container with id ffe0a1cf82a3f479313d71f9513e9baca836e1583f28494fc25d0ca2eba7d80e Jul 10 02:45:16.509412 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:16.509382 2585 generic.go:356] "Generic (PLEG): container finished" podID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerID="80f170552e90d2ca333b6dea079965dc2a6ff6af6672c89d92a5258f5f1db1e4" exitCode=0 Jul 10 02:45:16.509912 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:16.509453 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" event={"ID":"d2e36d62-d432-4f0c-a07c-20239d93c70c","Type":"ContainerDied","Data":"80f170552e90d2ca333b6dea079965dc2a6ff6af6672c89d92a5258f5f1db1e4"} Jul 10 02:45:16.510620 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:16.510601 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" event={"ID":"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3","Type":"ContainerStarted","Data":"ffe0a1cf82a3f479313d71f9513e9baca836e1583f28494fc25d0ca2eba7d80e"} Jul 10 02:45:16.906333 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:16.906245 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:17.005121 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:17.005094 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:17.517607 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:17.517574 2585 generic.go:356] "Generic (PLEG): container finished" podID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerID="4356cb1a372a9d2314b9f62b2097f11bd4319fd90ec21edd5f96dbddeb8737be" exitCode=0 Jul 10 02:45:17.517961 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:17.517632 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" event={"ID":"d2e36d62-d432-4f0c-a07c-20239d93c70c","Type":"ContainerDied","Data":"4356cb1a372a9d2314b9f62b2097f11bd4319fd90ec21edd5f96dbddeb8737be"} Jul 10 02:45:18.771885 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.771858 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:18.852741 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.852707 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-bundle\") pod \"d2e36d62-d432-4f0c-a07c-20239d93c70c\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " Jul 10 02:45:18.852867 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.852756 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-util\") pod \"d2e36d62-d432-4f0c-a07c-20239d93c70c\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " Jul 10 02:45:18.852867 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.852805 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqnl6\" (UniqueName: \"kubernetes.io/projected/d2e36d62-d432-4f0c-a07c-20239d93c70c-kube-api-access-tqnl6\") pod \"d2e36d62-d432-4f0c-a07c-20239d93c70c\" (UID: \"d2e36d62-d432-4f0c-a07c-20239d93c70c\") " Jul 10 02:45:18.853074 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.853047 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-bundle" (OuterVolumeSpecName: "bundle") pod "d2e36d62-d432-4f0c-a07c-20239d93c70c" (UID: "d2e36d62-d432-4f0c-a07c-20239d93c70c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:45:18.854850 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.854822 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e36d62-d432-4f0c-a07c-20239d93c70c-kube-api-access-tqnl6" (OuterVolumeSpecName: "kube-api-access-tqnl6") pod "d2e36d62-d432-4f0c-a07c-20239d93c70c" (UID: "d2e36d62-d432-4f0c-a07c-20239d93c70c"). InnerVolumeSpecName "kube-api-access-tqnl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:45:18.857337 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.857315 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-util" (OuterVolumeSpecName: "util") pod "d2e36d62-d432-4f0c-a07c-20239d93c70c" (UID: "d2e36d62-d432-4f0c-a07c-20239d93c70c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:45:18.954390 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.954357 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:18.954390 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.954387 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2e36d62-d432-4f0c-a07c-20239d93c70c-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:18.954390 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:18.954395 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqnl6\" (UniqueName: \"kubernetes.io/projected/d2e36d62-d432-4f0c-a07c-20239d93c70c-kube-api-access-tqnl6\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:19.529196 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:19.529165 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" Jul 10 02:45:19.529348 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:19.529162 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffqmhg" event={"ID":"d2e36d62-d432-4f0c-a07c-20239d93c70c","Type":"ContainerDied","Data":"9532b3f0bcacb63542b4a7e9859bf883d1c5acc48701c3ffc2b73d649bfd5f86"} Jul 10 02:45:19.529348 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:19.529284 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9532b3f0bcacb63542b4a7e9859bf883d1c5acc48701c3ffc2b73d649bfd5f86" Jul 10 02:45:19.530738 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:19.530709 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" event={"ID":"5a3600b1-9342-45e8-9e9a-9adc1e4c33a3","Type":"ContainerStarted","Data":"9995c4a3dd569810733872f6e7747598e8c67a917f6cb57b2f47b3023fac8020"} Jul 10 02:45:19.552194 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:19.552138 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-48d6h" podStartSLOduration=1.746572435 podStartE2EDuration="4.552119706s" totalImagesPullingTime="2.805547271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:19.5508559 +0000 UTC m=+326.259664457" watchObservedRunningTime="2026-07-10 02:45:19.552119706 +0000 UTC m=+326.260928241" Jul 10 02:45:20.007974 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.007943 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-99hwd"] Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008241 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="util" Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008256 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="util" Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008266 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="pull" Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008272 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="pull" Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008284 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="extract" Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008289 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="extract" Jul 10 02:45:20.008379 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.008351 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e36d62-d432-4f0c-a07c-20239d93c70c" containerName="extract" Jul 10 02:45:20.010286 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.010268 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.012895 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.012877 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mjrfv\"" Jul 10 02:45:20.019472 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.019446 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-99hwd"] Jul 10 02:45:20.063487 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.063459 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77dced88-d1f5-45d3-8152-c665fb76efa2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-99hwd\" (UID: \"77dced88-d1f5-45d3-8152-c665fb76efa2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.063643 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.063502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njxh\" (UniqueName: \"kubernetes.io/projected/77dced88-d1f5-45d3-8152-c665fb76efa2-kube-api-access-8njxh\") pod \"cert-manager-webhook-6486ffd997-99hwd\" (UID: \"77dced88-d1f5-45d3-8152-c665fb76efa2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.164093 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.164057 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8njxh\" (UniqueName: \"kubernetes.io/projected/77dced88-d1f5-45d3-8152-c665fb76efa2-kube-api-access-8njxh\") pod \"cert-manager-webhook-6486ffd997-99hwd\" (UID: \"77dced88-d1f5-45d3-8152-c665fb76efa2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.164237 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.164141 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77dced88-d1f5-45d3-8152-c665fb76efa2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-99hwd\" (UID: \"77dced88-d1f5-45d3-8152-c665fb76efa2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.172699 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.172674 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77dced88-d1f5-45d3-8152-c665fb76efa2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-99hwd\" (UID: \"77dced88-d1f5-45d3-8152-c665fb76efa2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.172855 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.172834 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njxh\" (UniqueName: \"kubernetes.io/projected/77dced88-d1f5-45d3-8152-c665fb76efa2-kube-api-access-8njxh\") pod \"cert-manager-webhook-6486ffd997-99hwd\" (UID: \"77dced88-d1f5-45d3-8152-c665fb76efa2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.319705 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.319582 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:20.446704 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.446678 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-99hwd"] Jul 10 02:45:20.448527 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:20.448497 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77dced88_d1f5_45d3_8152_c665fb76efa2.slice/crio-39b69ea13bb4ea05f2632b127fd365d8f78801fd9349ed12ebaa8a36d778e85e WatchSource:0}: Error finding container 39b69ea13bb4ea05f2632b127fd365d8f78801fd9349ed12ebaa8a36d778e85e: Status 404 returned error can't find the container with id 39b69ea13bb4ea05f2632b127fd365d8f78801fd9349ed12ebaa8a36d778e85e Jul 10 02:45:20.537677 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.537620 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" event={"ID":"77dced88-d1f5-45d3-8152-c665fb76efa2","Type":"ContainerStarted","Data":"d492765b77b55b30dd931e3c2555865c7f841f739e3a8d6f6906e694fe8c2dcc"} Jul 10 02:45:20.537840 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.537688 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" event={"ID":"77dced88-d1f5-45d3-8152-c665fb76efa2","Type":"ContainerStarted","Data":"39b69ea13bb4ea05f2632b127fd365d8f78801fd9349ed12ebaa8a36d778e85e"} Jul 10 02:45:20.557214 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.557162 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" podStartSLOduration=1.557142504 podStartE2EDuration="1.557142504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:20.556260317 +0000 UTC m=+327.265068844" watchObservedRunningTime="2026-07-10 02:45:20.557142504 +0000 UTC m=+327.265951038" Jul 10 02:45:20.740963 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.740924 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:20.835477 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:20.835442 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:21.443982 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:21.443943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:21.541471 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:21.541438 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:22.318501 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:22.318466 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:22.411535 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:22.411502 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:24.320989 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.320958 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj"] Jul 10 02:45:24.323606 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.323583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.326615 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.326590 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 02:45:24.327572 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.327556 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mr89r\"" Jul 10 02:45:24.327627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.327585 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 02:45:24.334075 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.334054 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj"] Jul 10 02:45:24.400711 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.400682 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bb7d3f1-20ec-4af3-a668-59d5176fa1e4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jfhsj\" (UID: \"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.400851 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.400724 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkmjm\" (UniqueName: \"kubernetes.io/projected/8bb7d3f1-20ec-4af3-a668-59d5176fa1e4-kube-api-access-jkmjm\") pod \"openshift-lws-operator-fd8ccff4c-jfhsj\" (UID: \"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.501249 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.501214 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkmjm\" (UniqueName: \"kubernetes.io/projected/8bb7d3f1-20ec-4af3-a668-59d5176fa1e4-kube-api-access-jkmjm\") pod \"openshift-lws-operator-fd8ccff4c-jfhsj\" (UID: \"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.501401 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.501317 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bb7d3f1-20ec-4af3-a668-59d5176fa1e4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jfhsj\" (UID: \"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.501727 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.501708 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8bb7d3f1-20ec-4af3-a668-59d5176fa1e4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jfhsj\" (UID: \"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.510375 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.510347 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkmjm\" (UniqueName: \"kubernetes.io/projected/8bb7d3f1-20ec-4af3-a668-59d5176fa1e4-kube-api-access-jkmjm\") pod \"openshift-lws-operator-fd8ccff4c-jfhsj\" (UID: \"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.633538 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.633451 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" Jul 10 02:45:24.761851 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:24.761777 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj"] Jul 10 02:45:24.763964 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:24.763940 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bb7d3f1_20ec_4af3_a668_59d5176fa1e4.slice/crio-d15a4a31ebb093b973717f6746eca32bed63f420ad7743eeb0f7dc7683824752 WatchSource:0}: Error finding container d15a4a31ebb093b973717f6746eca32bed63f420ad7743eeb0f7dc7683824752: Status 404 returned error can't find the container with id d15a4a31ebb093b973717f6746eca32bed63f420ad7743eeb0f7dc7683824752 Jul 10 02:45:25.562327 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:25.562277 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" event={"ID":"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4","Type":"ContainerStarted","Data":"d15a4a31ebb093b973717f6746eca32bed63f420ad7743eeb0f7dc7683824752"} Jul 10 02:45:26.568966 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:26.568927 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" event={"ID":"8bb7d3f1-20ec-4af3-a668-59d5176fa1e4","Type":"ContainerStarted","Data":"017067ac9996dc2cf7f2f2efea631088ff49df3aec65806d51ecbe78a6dd02d2"} Jul 10 02:45:26.588890 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:26.588833 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jfhsj" podStartSLOduration=0.92460646 podStartE2EDuration="2.588815966s" totalImagesPullingTime="1.664209506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:26.587140065 +0000 UTC m=+333.295948636" watchObservedRunningTime="2026-07-10 02:45:26.588815966 +0000 UTC m=+333.297624500" Jul 10 02:45:27.416755 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:27.416714 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:27.511485 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:27.511457 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:27.548502 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:27.548471 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-99hwd" Jul 10 02:45:27.655493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:27.655455 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:27.755955 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:27.755924 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:29.614699 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.614667 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:29.801006 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.800975 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm"] Jul 10 02:45:29.844351 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.844317 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm"] Jul 10 02:45:29.844501 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.844434 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:29.847505 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.847481 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 02:45:29.847618 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.847536 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 02:45:29.847684 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.847631 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-m8gzs\"" Jul 10 02:45:29.949535 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.949498 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:29.949737 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.949567 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qlfw\" (UniqueName: \"kubernetes.io/projected/4684d26d-3eb1-4d2b-9444-779873cc8668-kube-api-access-5qlfw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:29.949737 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:29.949699 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.050106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.050069 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qlfw\" (UniqueName: \"kubernetes.io/projected/4684d26d-3eb1-4d2b-9444-779873cc8668-kube-api-access-5qlfw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.050285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.050124 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.050285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.050153 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.050495 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.050479 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.050563 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.050544 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.059733 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.059704 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qlfw\" (UniqueName: \"kubernetes.io/projected/4684d26d-3eb1-4d2b-9444-779873cc8668-kube-api-access-5qlfw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.153464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.153435 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:30.295053 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.295030 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm"] Jul 10 02:45:30.297316 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:30.297285 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4684d26d_3eb1_4d2b_9444_779873cc8668.slice/crio-048933211a27658a9dc5b2876b02d5999dc6a9cf139e2b2e60eae549e765faa4 WatchSource:0}: Error finding container 048933211a27658a9dc5b2876b02d5999dc6a9cf139e2b2e60eae549e765faa4: Status 404 returned error can't find the container with id 048933211a27658a9dc5b2876b02d5999dc6a9cf139e2b2e60eae549e765faa4 Jul 10 02:45:30.588492 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.588402 2585 generic.go:356] "Generic (PLEG): container finished" podID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerID="6bf70028e1eb3dd0bed2f50b7f704087d3707468b591ae1caa181821fb18ef2c" exitCode=0 Jul 10 02:45:30.588681 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.588489 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" event={"ID":"4684d26d-3eb1-4d2b-9444-779873cc8668","Type":"ContainerDied","Data":"6bf70028e1eb3dd0bed2f50b7f704087d3707468b591ae1caa181821fb18ef2c"} Jul 10 02:45:30.588681 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:30.588532 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" event={"ID":"4684d26d-3eb1-4d2b-9444-779873cc8668","Type":"ContainerStarted","Data":"048933211a27658a9dc5b2876b02d5999dc6a9cf139e2b2e60eae549e765faa4"} Jul 10 02:45:32.599215 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:32.599174 2585 generic.go:356] "Generic (PLEG): container finished" podID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerID="f7ea20824e6efe94cf6dabee6216ae6185953d642c7af43a7a021025cbdda2be" exitCode=0 Jul 10 02:45:32.599679 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:32.599263 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" event={"ID":"4684d26d-3eb1-4d2b-9444-779873cc8668","Type":"ContainerDied","Data":"f7ea20824e6efe94cf6dabee6216ae6185953d642c7af43a7a021025cbdda2be"} Jul 10 02:45:33.606642 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:33.606604 2585 generic.go:356] "Generic (PLEG): container finished" podID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerID="c911231a3e6f3e74a45f342b8e5923d44a6ba7c7b1686a1b06f2ac7da5c34549" exitCode=0 Jul 10 02:45:33.607033 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:33.606672 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" event={"ID":"4684d26d-3eb1-4d2b-9444-779873cc8668","Type":"ContainerDied","Data":"c911231a3e6f3e74a45f342b8e5923d44a6ba7c7b1686a1b06f2ac7da5c34549"} Jul 10 02:45:34.738257 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.738235 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:34.792491 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.792457 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-util\") pod \"4684d26d-3eb1-4d2b-9444-779873cc8668\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " Jul 10 02:45:34.792666 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.792499 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-bundle\") pod \"4684d26d-3eb1-4d2b-9444-779873cc8668\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " Jul 10 02:45:34.792666 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.792622 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qlfw\" (UniqueName: \"kubernetes.io/projected/4684d26d-3eb1-4d2b-9444-779873cc8668-kube-api-access-5qlfw\") pod \"4684d26d-3eb1-4d2b-9444-779873cc8668\" (UID: \"4684d26d-3eb1-4d2b-9444-779873cc8668\") " Jul 10 02:45:34.793241 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.793210 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-bundle" (OuterVolumeSpecName: "bundle") pod "4684d26d-3eb1-4d2b-9444-779873cc8668" (UID: "4684d26d-3eb1-4d2b-9444-779873cc8668"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:45:34.794595 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.794570 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4684d26d-3eb1-4d2b-9444-779873cc8668-kube-api-access-5qlfw" (OuterVolumeSpecName: "kube-api-access-5qlfw") pod "4684d26d-3eb1-4d2b-9444-779873cc8668" (UID: "4684d26d-3eb1-4d2b-9444-779873cc8668"). InnerVolumeSpecName "kube-api-access-5qlfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:45:34.800425 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.800401 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-util" (OuterVolumeSpecName: "util") pod "4684d26d-3eb1-4d2b-9444-779873cc8668" (UID: "4684d26d-3eb1-4d2b-9444-779873cc8668"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:45:34.893706 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.893610 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:34.893706 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.893636 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4684d26d-3eb1-4d2b-9444-779873cc8668-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:34.893706 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:34.893662 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qlfw\" (UniqueName: \"kubernetes.io/projected/4684d26d-3eb1-4d2b-9444-779873cc8668-kube-api-access-5qlfw\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:45:35.617016 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:35.616986 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" Jul 10 02:45:35.617016 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:35.617002 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af82jlm" event={"ID":"4684d26d-3eb1-4d2b-9444-779873cc8668","Type":"ContainerDied","Data":"048933211a27658a9dc5b2876b02d5999dc6a9cf139e2b2e60eae549e765faa4"} Jul 10 02:45:35.617223 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:35.617042 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="048933211a27658a9dc5b2876b02d5999dc6a9cf139e2b2e60eae549e765faa4" Jul 10 02:45:48.708353 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:48.708276 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:48.823310 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:48.823271 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:49.514091 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.514060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:49.787328 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.787235 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:49.893792 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.893760 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:49.986065 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986027 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh"] Jul 10 02:45:49.986422 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986403 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="extract" Jul 10 02:45:49.986422 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986424 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="extract" Jul 10 02:45:49.986578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986444 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="util" Jul 10 02:45:49.986578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986451 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="util" Jul 10 02:45:49.986578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986463 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="pull" Jul 10 02:45:49.986578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986471 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="pull" Jul 10 02:45:49.986578 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.986552 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4684d26d-3eb1-4d2b-9444-779873cc8668" containerName="extract" Jul 10 02:45:49.988770 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.988750 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:49.994848 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.992811 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 02:45:49.994848 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.993017 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 02:45:49.994848 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.993065 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 02:45:49.995052 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.994936 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 02:45:49.995584 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:49.995544 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q69t7\"" Jul 10 02:45:50.012122 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.012090 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh"] Jul 10 02:45:50.124773 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.124697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjt49\" (UniqueName: \"kubernetes.io/projected/670a16a3-4588-4d54-836e-307166d53f89-kube-api-access-fjt49\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.124773 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.124752 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/670a16a3-4588-4d54-836e-307166d53f89-webhook-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.124773 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.124773 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/670a16a3-4588-4d54-836e-307166d53f89-tmp\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.125038 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.124847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/670a16a3-4588-4d54-836e-307166d53f89-manifests\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.125038 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.124936 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/670a16a3-4588-4d54-836e-307166d53f89-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.225357 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.225322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjt49\" (UniqueName: \"kubernetes.io/projected/670a16a3-4588-4d54-836e-307166d53f89-kube-api-access-fjt49\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.225561 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.225379 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/670a16a3-4588-4d54-836e-307166d53f89-webhook-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.225561 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.225399 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/670a16a3-4588-4d54-836e-307166d53f89-tmp\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.225561 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.225421 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/670a16a3-4588-4d54-836e-307166d53f89-manifests\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.225561 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.225474 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/670a16a3-4588-4d54-836e-307166d53f89-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.225927 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.225904 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/670a16a3-4588-4d54-836e-307166d53f89-manifests\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.227797 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.227772 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/670a16a3-4588-4d54-836e-307166d53f89-tmp\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.227887 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.227850 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/670a16a3-4588-4d54-836e-307166d53f89-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.227928 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.227884 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/670a16a3-4588-4d54-836e-307166d53f89-webhook-cert\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.241800 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.241774 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjt49\" (UniqueName: \"kubernetes.io/projected/670a16a3-4588-4d54-836e-307166d53f89-kube-api-access-fjt49\") pod \"opendatahub-operator-controller-manager-68647b69dd-d72rh\" (UID: \"670a16a3-4588-4d54-836e-307166d53f89\") " pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.300492 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.300456 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:50.305191 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.305158 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:50.449326 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:45:50.449298 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod670a16a3_4588_4d54_836e_307166d53f89.slice/crio-149387b50c9affd1af227d890e4f72c34d92ba2c0fe76682d8f8955b9b4f5218 WatchSource:0}: Error finding container 149387b50c9affd1af227d890e4f72c34d92ba2c0fe76682d8f8955b9b4f5218: Status 404 returned error can't find the container with id 149387b50c9affd1af227d890e4f72c34d92ba2c0fe76682d8f8955b9b4f5218 Jul 10 02:45:50.449602 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.449583 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh"] Jul 10 02:45:50.450952 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.450931 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:50.546748 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.546712 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:50.648679 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.648566 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:50.682892 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.682862 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" event={"ID":"670a16a3-4588-4d54-836e-307166d53f89","Type":"ContainerStarted","Data":"149387b50c9affd1af227d890e4f72c34d92ba2c0fe76682d8f8955b9b4f5218"} Jul 10 02:45:50.745310 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:50.745268 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:53.699194 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:53.699161 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" event={"ID":"670a16a3-4588-4d54-836e-307166d53f89","Type":"ContainerStarted","Data":"b075a1cacff0bc194867e457d2a06b5003ebe13bf016d36df327c077d833624e"} Jul 10 02:45:54.705299 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:54.705252 2585 generic.go:356] "Generic (PLEG): container finished" podID="670a16a3-4588-4d54-836e-307166d53f89" containerID="b075a1cacff0bc194867e457d2a06b5003ebe13bf016d36df327c077d833624e" exitCode=0 Jul 10 02:45:54.705676 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:54.705325 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" event={"ID":"670a16a3-4588-4d54-836e-307166d53f89","Type":"ContainerDied","Data":"b075a1cacff0bc194867e457d2a06b5003ebe13bf016d36df327c077d833624e"} Jul 10 02:45:55.712766 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:55.712735 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" event={"ID":"670a16a3-4588-4d54-836e-307166d53f89","Type":"ContainerStarted","Data":"76b81d6ac18bff27acd86e60a22d5947bbbfd9d81ccea02367a7590dbd670bad"} Jul 10 02:45:55.713189 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:55.712883 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:45:55.739351 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:55.739296 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" podStartSLOduration=2.333188306 podStartE2EDuration="6.739277631s" totalImagesPullingTime="3.388364505s" totalInitContainerRuntime="1.01772482s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:55.736826297 +0000 UTC m=+362.445634835" watchObservedRunningTime="2026-07-10 02:45:55.739277631 +0000 UTC m=+362.448086165" Jul 10 02:45:55.907733 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:55.907696 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:45:56.018773 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:45:56.018683 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:01.524217 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:01.524178 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:01.622622 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:01.622586 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:06.719021 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:06.718989 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68647b69dd-d72rh" Jul 10 02:46:09.875600 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:09.875565 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:10.011783 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.011737 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:10.487985 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.487951 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:10.636898 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.636862 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn"] Jul 10 02:46:10.645285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.645259 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.648370 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.648347 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 02:46:10.648493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.648386 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 02:46:10.648677 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.648658 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 02:46:10.648829 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.648812 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 02:46:10.649832 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.649817 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vjqd4\"" Jul 10 02:46:10.661912 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.661890 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn"] Jul 10 02:46:10.694593 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.694556 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qmvn\" (UniqueName: \"kubernetes.io/projected/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-kube-api-access-8qmvn\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.694776 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.694644 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-tmp\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.694776 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.694707 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-tls-certs\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.795922 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.795832 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qmvn\" (UniqueName: \"kubernetes.io/projected/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-kube-api-access-8qmvn\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.795922 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.795902 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-tmp\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.796158 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.795937 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-tls-certs\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.798129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.798096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-tmp\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.798320 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.798301 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-tls-certs\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.825041 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.825002 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qmvn\" (UniqueName: \"kubernetes.io/projected/cf17ac4b-09a5-4af0-b018-deb5b4c52b45-kube-api-access-8qmvn\") pod \"kube-auth-proxy-5d56d99d47-xr2qn\" (UID: \"cf17ac4b-09a5-4af0-b018-deb5b4c52b45\") " pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.871106 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.871053 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:10.957805 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.957774 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" Jul 10 02:46:10.965542 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:10.965491 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:11.066675 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:11.066606 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:11.097228 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:11.097201 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn"] Jul 10 02:46:11.098741 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:46:11.098697 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf17ac4b_09a5_4af0_b018_deb5b4c52b45.slice/crio-d1ccf2d5bf13744fc033dee274b75feaa3b433fd66f385294e7178c05475ecbe WatchSource:0}: Error finding container d1ccf2d5bf13744fc033dee274b75feaa3b433fd66f385294e7178c05475ecbe: Status 404 returned error can't find the container with id d1ccf2d5bf13744fc033dee274b75feaa3b433fd66f385294e7178c05475ecbe Jul 10 02:46:11.209323 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:11.209285 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:11.703451 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:11.703419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:11.788758 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:11.788716 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" event={"ID":"cf17ac4b-09a5-4af0-b018-deb5b4c52b45","Type":"ContainerStarted","Data":"d1ccf2d5bf13744fc033dee274b75feaa3b433fd66f385294e7178c05475ecbe"} Jul 10 02:46:11.882980 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:11.882930 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:12.237367 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:12.237328 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:12.327213 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:12.327175 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:13.492037 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:13.491994 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:14.810326 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:14.810291 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" event={"ID":"cf17ac4b-09a5-4af0-b018-deb5b4c52b45","Type":"ContainerStarted","Data":"74848b2d1f41bf394327d7f225313a4720e42089673e04ca1a0d67df9313f366"} Jul 10 02:46:17.803068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:17.803035 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:17.927868 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:17.927836 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:18.687204 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.687169 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:18.784370 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.784318 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d56d99d47-xr2qn" podStartSLOduration=5.994297249 podStartE2EDuration="8.784300369s" totalImagesPullingTime="2.79000312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:14.829983021 +0000 UTC m=+381.538791555" watchObservedRunningTime="2026-07-10 02:46:18.784300369 +0000 UTC m=+385.493108903" Jul 10 02:46:18.785223 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.785205 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll"] Jul 10 02:46:18.788375 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.788348 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.793293 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.793254 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 02:46:18.793427 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.793369 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 02:46:18.794055 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.794036 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 02:46:18.794269 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.794256 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6mzt5\"" Jul 10 02:46:18.794694 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.794680 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 02:46:18.818182 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.818153 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll"] Jul 10 02:46:18.854779 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.854744 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:18.870800 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.870773 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.870898 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.870879 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.870933 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.870903 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6fhm\" (UniqueName: \"kubernetes.io/projected/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-kube-api-access-w6fhm\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.870962 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.870932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.871000 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.870964 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.871032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.870993 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.871032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.871023 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972252 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972159 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972252 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972198 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6fhm\" (UniqueName: \"kubernetes.io/projected/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-kube-api-access-w6fhm\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972252 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972224 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972255 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972283 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.972532 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.972346 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.973062 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.973035 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.974634 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.974610 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.974799 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.974777 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.974990 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.974970 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.975055 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.974988 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.984080 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.984031 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:18.984181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:18.984089 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6fhm\" (UniqueName: \"kubernetes.io/projected/de76de0e-68e5-4522-8bf0-5f9dfb5ef663-kube-api-access-w6fhm\") pod \"istiod-openshift-gateway-94bb8fbfd-w6wll\" (UID: \"de76de0e-68e5-4522-8bf0-5f9dfb5ef663\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:19.046174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:19.046119 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:19.099178 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:19.099147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:19.319822 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:19.319715 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll"] Jul 10 02:46:19.322160 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:46:19.322134 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde76de0e_68e5_4522_8bf0_5f9dfb5ef663.slice/crio-da58eec8bb09aa7db9a0697b33ad677abd9b4a1d65aa249b539647136f7cf440 WatchSource:0}: Error finding container da58eec8bb09aa7db9a0697b33ad677abd9b4a1d65aa249b539647136f7cf440: Status 404 returned error can't find the container with id da58eec8bb09aa7db9a0697b33ad677abd9b4a1d65aa249b539647136f7cf440 Jul 10 02:46:19.835595 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:19.835556 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" event={"ID":"de76de0e-68e5-4522-8bf0-5f9dfb5ef663","Type":"ContainerStarted","Data":"da58eec8bb09aa7db9a0697b33ad677abd9b4a1d65aa249b539647136f7cf440"} Jul 10 02:46:22.076841 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.076799 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:22.139817 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.139783 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt"] Jul 10 02:46:22.147219 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.147193 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.149763 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.149735 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 02:46:22.150198 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.150176 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fz9qd\"" Jul 10 02:46:22.161091 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.161066 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt"] Jul 10 02:46:22.189528 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.189483 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:22.202696 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.202643 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.202875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.202754 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzlh5\" (UniqueName: \"kubernetes.io/projected/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-kube-api-access-jzlh5\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.303783 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.303742 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzlh5\" (UniqueName: \"kubernetes.io/projected/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-kube-api-access-jzlh5\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.303950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.303868 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.304010 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:46:22.303994 2585 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 02:46:22.304076 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:46:22.304065 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-cert podName:dcea5d17-55fb-42aa-bc6f-9d993ecebae0 nodeName:}" failed. No retries permitted until 2026-07-10 02:46:22.804045804 +0000 UTC m=+389.512854354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-cert") pod "llmisvc-controller-manager-777c6bd7d5-cz2zt" (UID: "dcea5d17-55fb-42aa-bc6f-9d993ecebae0") : secret "llmisvc-webhook-server-cert" not found Jul 10 02:46:22.315384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.315356 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzlh5\" (UniqueName: \"kubernetes.io/projected/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-kube-api-access-jzlh5\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.504625 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.504592 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:22.599477 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.599436 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 02:46:22.599542 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.599501 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 02:46:22.807274 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.807179 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.809633 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.809611 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcea5d17-55fb-42aa-bc6f-9d993ecebae0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cz2zt\" (UID: \"dcea5d17-55fb-42aa-bc6f-9d993ecebae0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:22.854255 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.854217 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" event={"ID":"de76de0e-68e5-4522-8bf0-5f9dfb5ef663","Type":"ContainerStarted","Data":"e22bf82273f916e76da2afd30cb4d4547468172001e245b580aa361598f43165"} Jul 10 02:46:22.854425 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:22.854361 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:23.060504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.060400 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:23.212202 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.212140 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" podStartSLOduration=1.9369631790000001 podStartE2EDuration="5.212118777s" totalImagesPullingTime="3.275155598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:22.881789781 +0000 UTC m=+389.590598313" watchObservedRunningTime="2026-07-10 02:46:23.212118777 +0000 UTC m=+389.920927315" Jul 10 02:46:23.213599 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.213581 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt"] Jul 10 02:46:23.215633 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:46:23.215606 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddcea5d17_55fb_42aa_bc6f_9d993ecebae0.slice/crio-8122f818e0ac714f81389b2b37f48898a1cbcfb6a4efa5f68440cd3ebc71f74b WatchSource:0}: Error finding container 8122f818e0ac714f81389b2b37f48898a1cbcfb6a4efa5f68440cd3ebc71f74b: Status 404 returned error can't find the container with id 8122f818e0ac714f81389b2b37f48898a1cbcfb6a4efa5f68440cd3ebc71f74b Jul 10 02:46:23.376117 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.376032 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:23.476246 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.476215 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:23.599270 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.599234 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:46:23.859946 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.859908 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" event={"ID":"dcea5d17-55fb-42aa-bc6f-9d993ecebae0","Type":"ContainerStarted","Data":"8122f818e0ac714f81389b2b37f48898a1cbcfb6a4efa5f68440cd3ebc71f74b"} Jul 10 02:46:23.861709 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:23.861687 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-w6wll" Jul 10 02:46:27.879863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:27.879829 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" event={"ID":"dcea5d17-55fb-42aa-bc6f-9d993ecebae0","Type":"ContainerStarted","Data":"a33b8cabb1994df962abbf9b596e48ea46f0062625b736d65a2a169a42c85e92"} Jul 10 02:46:27.880329 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:27.879932 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:46:27.897729 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:27.897676 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" podStartSLOduration=2.226092673 podStartE2EDuration="5.897641268s" totalImagesPullingTime="3.671548595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:27.895876493 +0000 UTC m=+394.604685028" watchObservedRunningTime="2026-07-10 02:46:27.897641268 +0000 UTC m=+394.606449801" Jul 10 02:46:58.886787 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:46:58.886754 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cz2zt" Jul 10 02:47:11.400763 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:11.400722 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:11.494135 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:11.494099 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:11.580726 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:11.580687 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:26.972071 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:26.972035 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf"] Jul 10 02:47:26.977737 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:26.976625 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:26.979858 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:26.979837 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xwllg\"" Jul 10 02:47:26.980163 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:26.980083 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 02:47:26.981507 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:26.981316 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 02:47:26.983849 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:26.983824 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf"] Jul 10 02:47:27.072129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.072081 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.072321 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.072139 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpmqs\" (UniqueName: \"kubernetes.io/projected/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-kube-api-access-cpmqs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.072321 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.072221 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.173049 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.173010 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpmqs\" (UniqueName: \"kubernetes.io/projected/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-kube-api-access-cpmqs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.173261 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.173075 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.173261 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.173154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.173458 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.173437 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.173511 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.173493 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.182151 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.182117 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpmqs\" (UniqueName: \"kubernetes.io/projected/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-kube-api-access-cpmqs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.289764 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.289673 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:27.423028 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.423002 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf"] Jul 10 02:47:27.424761 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:47:27.424730 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564b90d2_f39c_48a6_b2e6_4c02527c0bf9.slice/crio-e2e72080a1b8e2d64b89735867f6548bf7eb321f2ace954c99d41c51b966edb3 WatchSource:0}: Error finding container e2e72080a1b8e2d64b89735867f6548bf7eb321f2ace954c99d41c51b966edb3: Status 404 returned error can't find the container with id e2e72080a1b8e2d64b89735867f6548bf7eb321f2ace954c99d41c51b966edb3 Jul 10 02:47:27.552298 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.552231 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m"] Jul 10 02:47:27.555889 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.555872 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.564982 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.564957 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m"] Jul 10 02:47:27.678498 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.678459 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.678729 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.678595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.678729 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.678632 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds72h\" (UniqueName: \"kubernetes.io/projected/3c09e201-78a3-4db7-9613-d9d3ed0e9344-kube-api-access-ds72h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.779923 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.779884 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.779923 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.779929 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds72h\" (UniqueName: \"kubernetes.io/projected/3c09e201-78a3-4db7-9613-d9d3ed0e9344-kube-api-access-ds72h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.780112 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.779976 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.780284 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.780266 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.780370 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.780350 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.789836 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.789805 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds72h\" (UniqueName: \"kubernetes.io/projected/3c09e201-78a3-4db7-9613-d9d3ed0e9344-kube-api-access-ds72h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.900157 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.900077 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:27.954976 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.954945 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl"] Jul 10 02:47:27.959258 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.959234 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:27.969050 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:27.969024 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl"] Jul 10 02:47:28.042420 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.042396 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m"] Jul 10 02:47:28.043571 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:47:28.043546 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c09e201_78a3_4db7_9613_d9d3ed0e9344.slice/crio-059a0310158dc6840a9b27ccf8b514df03e553455874a6ea624a16dd7305ac80 WatchSource:0}: Error finding container 059a0310158dc6840a9b27ccf8b514df03e553455874a6ea624a16dd7305ac80: Status 404 returned error can't find the container with id 059a0310158dc6840a9b27ccf8b514df03e553455874a6ea624a16dd7305ac80 Jul 10 02:47:28.082601 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.082571 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.082716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.082616 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrsdx\" (UniqueName: \"kubernetes.io/projected/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-kube-api-access-qrsdx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.082716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.082700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.157814 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.157743 2585 generic.go:356] "Generic (PLEG): container finished" podID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerID="960af1c0821fb43ebc784523f96ba8ef83391efad58f4e2b77f3cfec2251766c" exitCode=0 Jul 10 02:47:28.157950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.157821 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" event={"ID":"3c09e201-78a3-4db7-9613-d9d3ed0e9344","Type":"ContainerDied","Data":"960af1c0821fb43ebc784523f96ba8ef83391efad58f4e2b77f3cfec2251766c"} Jul 10 02:47:28.157950 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.157870 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" event={"ID":"3c09e201-78a3-4db7-9613-d9d3ed0e9344","Type":"ContainerStarted","Data":"059a0310158dc6840a9b27ccf8b514df03e553455874a6ea624a16dd7305ac80"} Jul 10 02:47:28.159384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.159360 2585 generic.go:356] "Generic (PLEG): container finished" podID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerID="9c45a4b3f0c9ecf85790ca052012d65f0caaa8ffdb4d01340a516775495eeea5" exitCode=0 Jul 10 02:47:28.159493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.159429 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" event={"ID":"564b90d2-f39c-48a6-b2e6-4c02527c0bf9","Type":"ContainerDied","Data":"9c45a4b3f0c9ecf85790ca052012d65f0caaa8ffdb4d01340a516775495eeea5"} Jul 10 02:47:28.159493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.159475 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" event={"ID":"564b90d2-f39c-48a6-b2e6-4c02527c0bf9","Type":"ContainerStarted","Data":"e2e72080a1b8e2d64b89735867f6548bf7eb321f2ace954c99d41c51b966edb3"} Jul 10 02:47:28.183467 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.183437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.183598 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.183525 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.183598 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.183548 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrsdx\" (UniqueName: \"kubernetes.io/projected/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-kube-api-access-qrsdx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.183915 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.183887 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.183973 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.183949 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.192566 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.192541 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrsdx\" (UniqueName: \"kubernetes.io/projected/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-kube-api-access-qrsdx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.269459 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.269418 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:28.352723 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.352694 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh"] Jul 10 02:47:28.356476 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.356446 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.368535 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.368450 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh"] Jul 10 02:47:28.412029 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.412001 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl"] Jul 10 02:47:28.413959 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:47:28.413930 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e83868_33a2_4f7d_8d43_6fcdac3bb079.slice/crio-f8d0a61f7497cd745bbe1a028d10b5bdf140b441589adbfb6da7278b04e1bfe5 WatchSource:0}: Error finding container f8d0a61f7497cd745bbe1a028d10b5bdf140b441589adbfb6da7278b04e1bfe5: Status 404 returned error can't find the container with id f8d0a61f7497cd745bbe1a028d10b5bdf140b441589adbfb6da7278b04e1bfe5 Jul 10 02:47:28.487483 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.487450 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr95d\" (UniqueName: \"kubernetes.io/projected/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-kube-api-access-gr95d\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.487629 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.487518 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.487703 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.487620 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.588893 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.588850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.589067 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.588911 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr95d\" (UniqueName: \"kubernetes.io/projected/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-kube-api-access-gr95d\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.589067 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.588963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.589323 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.589304 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.589355 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.589313 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.598726 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.598695 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr95d\" (UniqueName: \"kubernetes.io/projected/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-kube-api-access-gr95d\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.669141 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.669055 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:28.817145 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:28.817123 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh"] Jul 10 02:47:28.819025 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:47:28.818995 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb3ef73a_b3fa_4faf_ba6f_e1f25315d9da.slice/crio-02d169ffc54e67d0cd27b62111d3ed33ae0145f28cf1f9587511ec488a5df3de WatchSource:0}: Error finding container 02d169ffc54e67d0cd27b62111d3ed33ae0145f28cf1f9587511ec488a5df3de: Status 404 returned error can't find the container with id 02d169ffc54e67d0cd27b62111d3ed33ae0145f28cf1f9587511ec488a5df3de Jul 10 02:47:29.166900 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:29.166871 2585 generic.go:356] "Generic (PLEG): container finished" podID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerID="c7af5b3f520a0a08e6cd4a9b6fc3025a4564b71d417ba853816815ce65b5ff44" exitCode=0 Jul 10 02:47:29.167288 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:29.167015 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" event={"ID":"d0e83868-33a2-4f7d-8d43-6fcdac3bb079","Type":"ContainerDied","Data":"c7af5b3f520a0a08e6cd4a9b6fc3025a4564b71d417ba853816815ce65b5ff44"} Jul 10 02:47:29.167288 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:29.167064 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" event={"ID":"d0e83868-33a2-4f7d-8d43-6fcdac3bb079","Type":"ContainerStarted","Data":"f8d0a61f7497cd745bbe1a028d10b5bdf140b441589adbfb6da7278b04e1bfe5"} Jul 10 02:47:29.169277 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:29.169147 2585 generic.go:356] "Generic (PLEG): container finished" podID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerID="97c45ad73d5fd2e7cceb77b9e95662d4bcf3ade587bd003d035cdd5af12ae3f8" exitCode=0 Jul 10 02:47:29.169277 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:29.169184 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" event={"ID":"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da","Type":"ContainerDied","Data":"97c45ad73d5fd2e7cceb77b9e95662d4bcf3ade587bd003d035cdd5af12ae3f8"} Jul 10 02:47:29.169277 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:29.169239 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" event={"ID":"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da","Type":"ContainerStarted","Data":"02d169ffc54e67d0cd27b62111d3ed33ae0145f28cf1f9587511ec488a5df3de"} Jul 10 02:47:30.178536 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:30.178504 2585 generic.go:356] "Generic (PLEG): container finished" podID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerID="56b7925238526ea1676b6df99634d71048926b03d95067808717fcbd36bd2c70" exitCode=0 Jul 10 02:47:30.178958 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:30.178582 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" event={"ID":"3c09e201-78a3-4db7-9613-d9d3ed0e9344","Type":"ContainerDied","Data":"56b7925238526ea1676b6df99634d71048926b03d95067808717fcbd36bd2c70"} Jul 10 02:47:30.180378 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:30.180362 2585 generic.go:356] "Generic (PLEG): container finished" podID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerID="e31d4839d5a7b45a0fe0ada211b3890be4213e1197aceef0148cadf1e1cec638" exitCode=0 Jul 10 02:47:30.180498 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:30.180455 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" event={"ID":"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da","Type":"ContainerDied","Data":"e31d4839d5a7b45a0fe0ada211b3890be4213e1197aceef0148cadf1e1cec638"} Jul 10 02:47:30.182335 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:30.182311 2585 generic.go:356] "Generic (PLEG): container finished" podID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerID="c2c590a19e99160137c3d0cbaf6b14a77a52f43739f04042fb21189da52f80bf" exitCode=0 Jul 10 02:47:30.182424 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:30.182353 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" event={"ID":"564b90d2-f39c-48a6-b2e6-4c02527c0bf9","Type":"ContainerDied","Data":"c2c590a19e99160137c3d0cbaf6b14a77a52f43739f04042fb21189da52f80bf"} Jul 10 02:47:31.189590 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.189556 2585 generic.go:356] "Generic (PLEG): container finished" podID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerID="467a6ad32e2b5c6b6a0f8fce61c21329cfc7c28e846cc42c56b1a24c25ec303c" exitCode=0 Jul 10 02:47:31.190078 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.189643 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" event={"ID":"d0e83868-33a2-4f7d-8d43-6fcdac3bb079","Type":"ContainerDied","Data":"467a6ad32e2b5c6b6a0f8fce61c21329cfc7c28e846cc42c56b1a24c25ec303c"} Jul 10 02:47:31.191939 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.191919 2585 generic.go:356] "Generic (PLEG): container finished" podID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerID="413ebb6cfc327cb938a8abcc0d02198d7a062db34b6c628914534e22e93a1abe" exitCode=0 Jul 10 02:47:31.192021 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.191986 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" event={"ID":"3c09e201-78a3-4db7-9613-d9d3ed0e9344","Type":"ContainerDied","Data":"413ebb6cfc327cb938a8abcc0d02198d7a062db34b6c628914534e22e93a1abe"} Jul 10 02:47:31.193933 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.193912 2585 generic.go:356] "Generic (PLEG): container finished" podID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerID="1edae5b524301894187a8647b0e0ac7b5ac62f1a6560e7a2979f5e33ea025e5b" exitCode=0 Jul 10 02:47:31.194023 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.193993 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" event={"ID":"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da","Type":"ContainerDied","Data":"1edae5b524301894187a8647b0e0ac7b5ac62f1a6560e7a2979f5e33ea025e5b"} Jul 10 02:47:31.196009 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.195986 2585 generic.go:356] "Generic (PLEG): container finished" podID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerID="8fafac9376c5529707f384d87d2554e56222cf5bf3058af197389933edcc0145" exitCode=0 Jul 10 02:47:31.196093 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:31.196047 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" event={"ID":"564b90d2-f39c-48a6-b2e6-4c02527c0bf9","Type":"ContainerDied","Data":"8fafac9376c5529707f384d87d2554e56222cf5bf3058af197389933edcc0145"} Jul 10 02:47:32.203420 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.203380 2585 generic.go:356] "Generic (PLEG): container finished" podID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerID="39c0f8bf7c97c5123c9ffc6abab78c16f405c8bd3fae85deab8a343cc9b7de4b" exitCode=0 Jul 10 02:47:32.203933 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.203468 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" event={"ID":"d0e83868-33a2-4f7d-8d43-6fcdac3bb079","Type":"ContainerDied","Data":"39c0f8bf7c97c5123c9ffc6abab78c16f405c8bd3fae85deab8a343cc9b7de4b"} Jul 10 02:47:32.355085 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.355047 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:32.401487 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.401461 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:32.404354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.404328 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:32.532093 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532056 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds72h\" (UniqueName: \"kubernetes.io/projected/3c09e201-78a3-4db7-9613-d9d3ed0e9344-kube-api-access-ds72h\") pod \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " Jul 10 02:47:32.532276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532119 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-bundle\") pod \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " Jul 10 02:47:32.532276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532151 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-util\") pod \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " Jul 10 02:47:32.532276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532232 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-util\") pod \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " Jul 10 02:47:32.532276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532257 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-util\") pod \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " Jul 10 02:47:32.532479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532289 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-bundle\") pod \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " Jul 10 02:47:32.532479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532322 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpmqs\" (UniqueName: \"kubernetes.io/projected/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-kube-api-access-cpmqs\") pod \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\" (UID: \"564b90d2-f39c-48a6-b2e6-4c02527c0bf9\") " Jul 10 02:47:32.532479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532345 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-bundle\") pod \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\" (UID: \"3c09e201-78a3-4db7-9613-d9d3ed0e9344\") " Jul 10 02:47:32.532479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532373 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr95d\" (UniqueName: \"kubernetes.io/projected/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-kube-api-access-gr95d\") pod \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\" (UID: \"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da\") " Jul 10 02:47:32.532906 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.532813 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-bundle" (OuterVolumeSpecName: "bundle") pod "eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" (UID: "eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:32.533420 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.533383 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-bundle" (OuterVolumeSpecName: "bundle") pod "564b90d2-f39c-48a6-b2e6-4c02527c0bf9" (UID: "564b90d2-f39c-48a6-b2e6-4c02527c0bf9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:32.533531 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.533482 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-bundle" (OuterVolumeSpecName: "bundle") pod "3c09e201-78a3-4db7-9613-d9d3ed0e9344" (UID: "3c09e201-78a3-4db7-9613-d9d3ed0e9344"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:32.534728 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.534698 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c09e201-78a3-4db7-9613-d9d3ed0e9344-kube-api-access-ds72h" (OuterVolumeSpecName: "kube-api-access-ds72h") pod "3c09e201-78a3-4db7-9613-d9d3ed0e9344" (UID: "3c09e201-78a3-4db7-9613-d9d3ed0e9344"). InnerVolumeSpecName "kube-api-access-ds72h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:32.534919 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.534895 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-kube-api-access-gr95d" (OuterVolumeSpecName: "kube-api-access-gr95d") pod "eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" (UID: "eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da"). InnerVolumeSpecName "kube-api-access-gr95d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:32.535104 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.535090 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-kube-api-access-cpmqs" (OuterVolumeSpecName: "kube-api-access-cpmqs") pod "564b90d2-f39c-48a6-b2e6-4c02527c0bf9" (UID: "564b90d2-f39c-48a6-b2e6-4c02527c0bf9"). InnerVolumeSpecName "kube-api-access-cpmqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:32.539207 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.539144 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-util" (OuterVolumeSpecName: "util") pod "564b90d2-f39c-48a6-b2e6-4c02527c0bf9" (UID: "564b90d2-f39c-48a6-b2e6-4c02527c0bf9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:32.541381 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.541357 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-util" (OuterVolumeSpecName: "util") pod "3c09e201-78a3-4db7-9613-d9d3ed0e9344" (UID: "3c09e201-78a3-4db7-9613-d9d3ed0e9344"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:32.541628 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.541603 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-util" (OuterVolumeSpecName: "util") pod "eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" (UID: "eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:32.633088 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633051 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633088 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633084 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633113 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633124 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpmqs\" (UniqueName: \"kubernetes.io/projected/564b90d2-f39c-48a6-b2e6-4c02527c0bf9-kube-api-access-cpmqs\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633134 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c09e201-78a3-4db7-9613-d9d3ed0e9344-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633143 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr95d\" (UniqueName: \"kubernetes.io/projected/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-kube-api-access-gr95d\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633151 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds72h\" (UniqueName: \"kubernetes.io/projected/3c09e201-78a3-4db7-9613-d9d3ed0e9344-kube-api-access-ds72h\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633159 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:32.633285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:32.633166 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:33.211291 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.211255 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" event={"ID":"3c09e201-78a3-4db7-9613-d9d3ed0e9344","Type":"ContainerDied","Data":"059a0310158dc6840a9b27ccf8b514df03e553455874a6ea624a16dd7305ac80"} Jul 10 02:47:33.211291 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.211272 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m" Jul 10 02:47:33.211291 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.211299 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="059a0310158dc6840a9b27ccf8b514df03e553455874a6ea624a16dd7305ac80" Jul 10 02:47:33.213247 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.213219 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" Jul 10 02:47:33.213381 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.213222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh" event={"ID":"eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da","Type":"ContainerDied","Data":"02d169ffc54e67d0cd27b62111d3ed33ae0145f28cf1f9587511ec488a5df3de"} Jul 10 02:47:33.213381 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.213329 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02d169ffc54e67d0cd27b62111d3ed33ae0145f28cf1f9587511ec488a5df3de" Jul 10 02:47:33.214983 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.214963 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" Jul 10 02:47:33.214983 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.214972 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf" event={"ID":"564b90d2-f39c-48a6-b2e6-4c02527c0bf9","Type":"ContainerDied","Data":"e2e72080a1b8e2d64b89735867f6548bf7eb321f2ace954c99d41c51b966edb3"} Jul 10 02:47:33.215112 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.215011 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2e72080a1b8e2d64b89735867f6548bf7eb321f2ace954c99d41c51b966edb3" Jul 10 02:47:33.348593 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.348564 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:33.541816 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.541723 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-bundle\") pod \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " Jul 10 02:47:33.541816 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.541787 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrsdx\" (UniqueName: \"kubernetes.io/projected/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-kube-api-access-qrsdx\") pod \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " Jul 10 02:47:33.541816 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.541817 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-util\") pod \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\" (UID: \"d0e83868-33a2-4f7d-8d43-6fcdac3bb079\") " Jul 10 02:47:33.542302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.542268 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-bundle" (OuterVolumeSpecName: "bundle") pod "d0e83868-33a2-4f7d-8d43-6fcdac3bb079" (UID: "d0e83868-33a2-4f7d-8d43-6fcdac3bb079"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:33.544007 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.543980 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-kube-api-access-qrsdx" (OuterVolumeSpecName: "kube-api-access-qrsdx") pod "d0e83868-33a2-4f7d-8d43-6fcdac3bb079" (UID: "d0e83868-33a2-4f7d-8d43-6fcdac3bb079"). InnerVolumeSpecName "kube-api-access-qrsdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:47:33.547248 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.547219 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-util" (OuterVolumeSpecName: "util") pod "d0e83868-33a2-4f7d-8d43-6fcdac3bb079" (UID: "d0e83868-33a2-4f7d-8d43-6fcdac3bb079"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:47:33.643273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.643237 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:33.643273 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.643270 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrsdx\" (UniqueName: \"kubernetes.io/projected/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-kube-api-access-qrsdx\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:33.643489 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:33.643284 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0e83868-33a2-4f7d-8d43-6fcdac3bb079-util\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:47:34.221588 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:34.221556 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" Jul 10 02:47:34.221588 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:34.221579 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl" event={"ID":"d0e83868-33a2-4f7d-8d43-6fcdac3bb079","Type":"ContainerDied","Data":"f8d0a61f7497cd745bbe1a028d10b5bdf140b441589adbfb6da7278b04e1bfe5"} Jul 10 02:47:34.222111 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:34.221621 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8d0a61f7497cd745bbe1a028d10b5bdf140b441589adbfb6da7278b04e1bfe5" Jul 10 02:47:40.031018 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:40.030981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:40.581626 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:40.581572 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:41.604921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:41.604886 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:44.069492 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:44.069453 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:44.177599 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:44.177563 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:44.268886 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:44.268850 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:44.762604 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:44.762567 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:44.863846 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:44.863810 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:45.189193 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:45.189155 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:46.890080 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890047 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf"] Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890366 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="pull" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890377 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="pull" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890384 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890389 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890397 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="util" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890402 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="util" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890408 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890413 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890419 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="util" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890424 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="util" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890433 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890438 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890448 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="pull" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890453 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="pull" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890462 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="util" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890467 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="util" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890473 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="extract" Jul 10 02:47:46.890478 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890479 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="extract" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890496 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="util" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890501 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="util" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890507 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="pull" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890512 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="pull" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890518 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="pull" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890522 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="pull" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890572 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e83868-33a2-4f7d-8d43-6fcdac3bb079" containerName="extract" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890581 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c09e201-78a3-4db7-9613-d9d3ed0e9344" containerName="extract" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890586 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da" containerName="extract" Jul 10 02:47:46.891068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.890592 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="564b90d2-f39c-48a6-b2e6-4c02527c0bf9" containerName="extract" Jul 10 02:47:46.893511 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.893495 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:47:46.897682 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.897662 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 02:47:46.897804 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.897715 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 02:47:46.898307 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.898287 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 02:47:46.898401 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.898370 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ljnpp\"" Jul 10 02:47:46.913392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.909634 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf"] Jul 10 02:47:46.953464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:46.953430 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkptq\" (UniqueName: \"kubernetes.io/projected/f40b5f77-9a6e-43da-a040-938da2abdbb2-kube-api-access-nkptq\") pod \"dns-operator-controller-manager-864dd78c4c-wwvmf\" (UID: \"f40b5f77-9a6e-43da-a040-938da2abdbb2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:47:47.054873 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:47.054835 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkptq\" (UniqueName: \"kubernetes.io/projected/f40b5f77-9a6e-43da-a040-938da2abdbb2-kube-api-access-nkptq\") pod \"dns-operator-controller-manager-864dd78c4c-wwvmf\" (UID: \"f40b5f77-9a6e-43da-a040-938da2abdbb2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:47:47.068443 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:47.068413 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkptq\" (UniqueName: \"kubernetes.io/projected/f40b5f77-9a6e-43da-a040-938da2abdbb2-kube-api-access-nkptq\") pod \"dns-operator-controller-manager-864dd78c4c-wwvmf\" (UID: \"f40b5f77-9a6e-43da-a040-938da2abdbb2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:47:47.202542 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:47.202496 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:47:47.346335 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:47.346295 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf"] Jul 10 02:47:47.347390 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:47:47.347358 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40b5f77_9a6e_43da_a040_938da2abdbb2.slice/crio-00f635d5f5ee6a18ec2693d0feadff7a27410999337ed371630d93081040be0b WatchSource:0}: Error finding container 00f635d5f5ee6a18ec2693d0feadff7a27410999337ed371630d93081040be0b: Status 404 returned error can't find the container with id 00f635d5f5ee6a18ec2693d0feadff7a27410999337ed371630d93081040be0b Jul 10 02:47:47.437129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:47.437088 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:47.682893 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:47.682864 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:48.300753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:48.300720 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" event={"ID":"f40b5f77-9a6e-43da-a040-938da2abdbb2","Type":"ContainerStarted","Data":"00f635d5f5ee6a18ec2693d0feadff7a27410999337ed371630d93081040be0b"} Jul 10 02:47:48.649859 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:48.649701 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:49.273112 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:49.273075 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:49.382087 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:49.382052 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:50.313729 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.313692 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" event={"ID":"f40b5f77-9a6e-43da-a040-938da2abdbb2","Type":"ContainerStarted","Data":"c432e3853adbb850480796e04f48d19bd6949de49cdda7e6800fc9c23db6ddbd"} Jul 10 02:47:50.313908 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.313786 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:47:50.341968 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.341920 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" podStartSLOduration=2.246546759 podStartE2EDuration="4.341903667s" totalImagesPullingTime="2.095356908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:50.340119772 +0000 UTC m=+477.048928309" watchObservedRunningTime="2026-07-10 02:47:50.341903667 +0000 UTC m=+477.050712201" Jul 10 02:47:50.871406 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.871372 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc"] Jul 10 02:47:50.874981 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.874964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:50.878082 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.878067 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4l7tn\"" Jul 10 02:47:50.889300 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.889274 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc"] Jul 10 02:47:50.991494 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.991462 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d57fba-1f0a-4112-b37c-ec43d038626b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:50.991717 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:50.991608 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnlp5\" (UniqueName: \"kubernetes.io/projected/21d57fba-1f0a-4112-b37c-ec43d038626b-kube-api-access-vnlp5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:51.092394 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.092357 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnlp5\" (UniqueName: \"kubernetes.io/projected/21d57fba-1f0a-4112-b37c-ec43d038626b-kube-api-access-vnlp5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:51.092394 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.092397 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d57fba-1f0a-4112-b37c-ec43d038626b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:51.092817 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.092793 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d57fba-1f0a-4112-b37c-ec43d038626b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:51.102736 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.102718 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnlp5\" (UniqueName: \"kubernetes.io/projected/21d57fba-1f0a-4112-b37c-ec43d038626b-kube-api-access-vnlp5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:51.184961 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.184933 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:51.318948 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.318922 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc"] Jul 10 02:47:51.321993 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:47:51.321966 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d57fba_1f0a_4112_b37c_ec43d038626b.slice/crio-09d785c6b6e4f333be559486cbffe778105dcf3a026789e6e10ca5cd7c5947bf WatchSource:0}: Error finding container 09d785c6b6e4f333be559486cbffe778105dcf3a026789e6e10ca5cd7c5947bf: Status 404 returned error can't find the container with id 09d785c6b6e4f333be559486cbffe778105dcf3a026789e6e10ca5cd7c5947bf Jul 10 02:47:51.378824 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:51.378786 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:52.328419 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:52.328373 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" event={"ID":"21d57fba-1f0a-4112-b37c-ec43d038626b","Type":"ContainerStarted","Data":"09d785c6b6e4f333be559486cbffe778105dcf3a026789e6e10ca5cd7c5947bf"} Jul 10 02:47:52.429354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:52.429311 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:54.385905 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:54.385866 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:54.488007 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:54.487972 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:54.787342 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:54.787287 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:54.854291 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:54.854255 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:55.084910 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:55.084819 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688c6dc994-z8m27"] Jul 10 02:47:55.219453 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:55.219388 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:55.318594 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:55.318542 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:55.419635 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:55.419549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:58.362323 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:58.362284 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" event={"ID":"21d57fba-1f0a-4112-b37c-ec43d038626b","Type":"ContainerStarted","Data":"4f294c7fa8c1cdbb36140347abd0c70f36eea7279782b20005885aadb0bbe1f0"} Jul 10 02:47:58.362810 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:58.362454 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:47:58.384957 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:58.384897 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" podStartSLOduration=2.1828994 podStartE2EDuration="8.384878343s" totalImagesPullingTime="6.201978943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:58.383196793 +0000 UTC m=+485.092005363" watchObservedRunningTime="2026-07-10 02:47:58.384878343 +0000 UTC m=+485.093686877" Jul 10 02:47:58.630332 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:58.630243 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:58.735146 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:58.735094 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:59.376480 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:59.376445 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:47:59.439945 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:47:59.439911 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:00.173901 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:00.173869 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:00.363748 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:00.363704 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:01.321138 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:01.321105 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-wwvmf" Jul 10 02:48:03.983597 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:03.983559 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:04.084092 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:04.084055 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:08.162746 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:08.162694 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:08.297427 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:08.297391 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:09.369792 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:09.369761 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:10.035951 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.035907 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:10.134164 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.134129 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:10.239095 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.239060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:10.334096 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.334015 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:10.554005 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.553966 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9"] Jul 10 02:48:10.559003 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.558977 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:10.581713 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.581687 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9"] Jul 10 02:48:10.675135 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.675037 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9"] Jul 10 02:48:10.675346 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:10.675324 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-g6zxz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" Jul 10 02:48:10.680957 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.680924 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6zxz\" (UniqueName: \"kubernetes.io/projected/b605dd06-b5c2-4286-8621-df5525e80e84-kube-api-access-g6zxz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:10.681035 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.680987 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b605dd06-b5c2-4286-8621-df5525e80e84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:10.782304 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.782254 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6zxz\" (UniqueName: \"kubernetes.io/projected/b605dd06-b5c2-4286-8621-df5525e80e84-kube-api-access-g6zxz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:10.782511 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.782330 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b605dd06-b5c2-4286-8621-df5525e80e84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:10.782790 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.782769 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b605dd06-b5c2-4286-8621-df5525e80e84-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:10.793032 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:10.793011 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6zxz\" (UniqueName: \"kubernetes.io/projected/b605dd06-b5c2-4286-8621-df5525e80e84-kube-api-access-g6zxz\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:11.209512 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.209471 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9"] Jul 10 02:48:11.220673 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.220616 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld"] Jul 10 02:48:11.224153 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.224132 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9"] Jul 10 02:48:11.224267 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.224224 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.231885 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.231865 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc"] Jul 10 02:48:11.232073 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.232052 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" containerName="manager" containerID="cri-o://4f294c7fa8c1cdbb36140347abd0c70f36eea7279782b20005885aadb0bbe1f0" gracePeriod=2 Jul 10 02:48:11.238353 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.238332 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld"] Jul 10 02:48:11.245310 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.245287 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc"] Jul 10 02:48:11.259245 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.259137 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl"] Jul 10 02:48:11.259639 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.259605 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" containerName="manager" Jul 10 02:48:11.259639 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.259629 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" containerName="manager" Jul 10 02:48:11.259808 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.259748 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" containerName="manager" Jul 10 02:48:11.263820 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.263799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.266960 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.266928 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:11.280342 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.280309 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl"] Jul 10 02:48:11.300043 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.300004 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:11.387880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.387835 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tht7k\" (UniqueName: \"kubernetes.io/projected/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-kube-api-access-tht7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rspsl\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.388059 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.387897 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/741c3d57-db62-4f82-8881-3869f3f87a75-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-p5cld\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.388059 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.387974 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rspsl\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.388059 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.388024 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qjdt\" (UniqueName: \"kubernetes.io/projected/741c3d57-db62-4f82-8881-3869f3f87a75-kube-api-access-5qjdt\") pod \"kuadrant-operator-controller-manager-55467d9f7-p5cld\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.430276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.430248 2585 generic.go:356] "Generic (PLEG): container finished" podID="21d57fba-1f0a-4112-b37c-ec43d038626b" containerID="4f294c7fa8c1cdbb36140347abd0c70f36eea7279782b20005885aadb0bbe1f0" exitCode=0 Jul 10 02:48:11.430425 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.430418 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:11.433178 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.433146 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:11.435427 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.435395 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:11.437126 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.437110 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:11.439555 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.439532 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:11.441584 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.441565 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:11.483396 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.483371 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:11.485988 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.485959 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:11.488243 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:11.488221 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-q4bzc\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:11.488533 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.488511 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rspsl\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.488573 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.488558 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qjdt\" (UniqueName: \"kubernetes.io/projected/741c3d57-db62-4f82-8881-3869f3f87a75-kube-api-access-5qjdt\") pod \"kuadrant-operator-controller-manager-55467d9f7-p5cld\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.488614 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.488599 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tht7k\" (UniqueName: \"kubernetes.io/projected/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-kube-api-access-tht7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rspsl\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.488667 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.488641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/741c3d57-db62-4f82-8881-3869f3f87a75-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-p5cld\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.488927 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.488909 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rspsl\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.488981 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.488953 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/741c3d57-db62-4f82-8881-3869f3f87a75-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-p5cld\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.509495 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.509467 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tht7k\" (UniqueName: \"kubernetes.io/projected/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-kube-api-access-tht7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rspsl\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.511311 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.511294 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qjdt\" (UniqueName: \"kubernetes.io/projected/741c3d57-db62-4f82-8881-3869f3f87a75-kube-api-access-5qjdt\") pod \"kuadrant-operator-controller-manager-55467d9f7-p5cld\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.535180 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.535149 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:11.589447 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.589390 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d57fba-1f0a-4112-b37c-ec43d038626b-extensions-socket-volume\") pod \"21d57fba-1f0a-4112-b37c-ec43d038626b\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " Jul 10 02:48:11.589881 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.589451 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b605dd06-b5c2-4286-8621-df5525e80e84-extensions-socket-volume\") pod \"b605dd06-b5c2-4286-8621-df5525e80e84\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " Jul 10 02:48:11.589881 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.589523 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnlp5\" (UniqueName: \"kubernetes.io/projected/21d57fba-1f0a-4112-b37c-ec43d038626b-kube-api-access-vnlp5\") pod \"21d57fba-1f0a-4112-b37c-ec43d038626b\" (UID: \"21d57fba-1f0a-4112-b37c-ec43d038626b\") " Jul 10 02:48:11.589881 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.589547 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6zxz\" (UniqueName: \"kubernetes.io/projected/b605dd06-b5c2-4286-8621-df5525e80e84-kube-api-access-g6zxz\") pod \"b605dd06-b5c2-4286-8621-df5525e80e84\" (UID: \"b605dd06-b5c2-4286-8621-df5525e80e84\") " Jul 10 02:48:11.590055 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.589943 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d57fba-1f0a-4112-b37c-ec43d038626b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "21d57fba-1f0a-4112-b37c-ec43d038626b" (UID: "21d57fba-1f0a-4112-b37c-ec43d038626b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:48:11.590259 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.590229 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b605dd06-b5c2-4286-8621-df5525e80e84-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b605dd06-b5c2-4286-8621-df5525e80e84" (UID: "b605dd06-b5c2-4286-8621-df5525e80e84"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:48:11.592076 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.592049 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d57fba-1f0a-4112-b37c-ec43d038626b-kube-api-access-vnlp5" (OuterVolumeSpecName: "kube-api-access-vnlp5") pod "21d57fba-1f0a-4112-b37c-ec43d038626b" (UID: "21d57fba-1f0a-4112-b37c-ec43d038626b"). InnerVolumeSpecName "kube-api-access-vnlp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:48:11.592492 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.592476 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b605dd06-b5c2-4286-8621-df5525e80e84-kube-api-access-g6zxz" (OuterVolumeSpecName: "kube-api-access-g6zxz") pod "b605dd06-b5c2-4286-8621-df5525e80e84" (UID: "b605dd06-b5c2-4286-8621-df5525e80e84"). InnerVolumeSpecName "kube-api-access-g6zxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:48:11.605734 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.605708 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:11.681949 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.681915 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld"] Jul 10 02:48:11.683303 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:48:11.683267 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741c3d57_db62_4f82_8881_3869f3f87a75.slice/crio-2a9b40d6099526764c2bf6ee298b9e251accf6d2b60f50b83a4906b09ad20bda WatchSource:0}: Error finding container 2a9b40d6099526764c2bf6ee298b9e251accf6d2b60f50b83a4906b09ad20bda: Status 404 returned error can't find the container with id 2a9b40d6099526764c2bf6ee298b9e251accf6d2b60f50b83a4906b09ad20bda Jul 10 02:48:11.690930 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.690907 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21d57fba-1f0a-4112-b37c-ec43d038626b-extensions-socket-volume\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:11.690930 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.690930 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b605dd06-b5c2-4286-8621-df5525e80e84-extensions-socket-volume\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:11.691091 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.690940 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnlp5\" (UniqueName: \"kubernetes.io/projected/21d57fba-1f0a-4112-b37c-ec43d038626b-kube-api-access-vnlp5\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:11.691091 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.690949 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6zxz\" (UniqueName: \"kubernetes.io/projected/b605dd06-b5c2-4286-8621-df5525e80e84-kube-api-access-g6zxz\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:11.766000 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:11.765974 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl"] Jul 10 02:48:11.768204 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:48:11.768179 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef270adf_55e7_4abb_aa61_3b2bb7db6e40.slice/crio-655144bc6303a11237ddf530a2ec511d1413de7dcd0f3634f77e78166debf217 WatchSource:0}: Error finding container 655144bc6303a11237ddf530a2ec511d1413de7dcd0f3634f77e78166debf217: Status 404 returned error can't find the container with id 655144bc6303a11237ddf530a2ec511d1413de7dcd0f3634f77e78166debf217 Jul 10 02:48:12.112105 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.112034 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d57fba-1f0a-4112-b37c-ec43d038626b" path="/var/lib/kubelet/pods/21d57fba-1f0a-4112-b37c-ec43d038626b/volumes" Jul 10 02:48:12.112384 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.112372 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" path="/var/lib/kubelet/pods/b605dd06-b5c2-4286-8621-df5525e80e84/volumes" Jul 10 02:48:12.436974 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.436936 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" event={"ID":"ef270adf-55e7-4abb-aa61-3b2bb7db6e40","Type":"ContainerStarted","Data":"bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253"} Jul 10 02:48:12.437176 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.436998 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" event={"ID":"ef270adf-55e7-4abb-aa61-3b2bb7db6e40","Type":"ContainerStarted","Data":"655144bc6303a11237ddf530a2ec511d1413de7dcd0f3634f77e78166debf217"} Jul 10 02:48:12.437176 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.437032 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:12.438285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.438262 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-q4bzc" Jul 10 02:48:12.438414 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.438292 2585 scope.go:117] "RemoveContainer" containerID="4f294c7fa8c1cdbb36140347abd0c70f36eea7279782b20005885aadb0bbe1f0" Jul 10 02:48:12.440054 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.440031 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:12.440163 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.440033 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" event={"ID":"741c3d57-db62-4f82-8881-3869f3f87a75","Type":"ContainerStarted","Data":"a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413"} Jul 10 02:48:12.440227 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.440159 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" event={"ID":"741c3d57-db62-4f82-8881-3869f3f87a75","Type":"ContainerStarted","Data":"2a9b40d6099526764c2bf6ee298b9e251accf6d2b60f50b83a4906b09ad20bda"} Jul 10 02:48:12.460732 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.460679 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" podStartSLOduration=1.460639881 podStartE2EDuration="1.460639881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:12.458791147 +0000 UTC m=+499.167599681" watchObservedRunningTime="2026-07-10 02:48:12.460639881 +0000 UTC m=+499.169448417" Jul 10 02:48:12.487555 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:12.487510 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:12.489965 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:12.489923 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" podStartSLOduration=1.4899063670000001 podStartE2EDuration="1.489906367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:12.484173453 +0000 UTC m=+499.192981988" watchObservedRunningTime="2026-07-10 02:48:12.489906367 +0000 UTC m=+499.198714901" Jul 10 02:48:13.446598 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:13.446570 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:14.023500 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:14.023466 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:14.110994 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:14.110956 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8zzg9\" is forbidden: User \"system:node:ip-10-0-139-175.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-175.ec2.internal' and this object" podUID="b605dd06-b5c2-4286-8621-df5525e80e84" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8zzg9" Jul 10 02:48:14.123632 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:14.123599 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:14.618118 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:14.618082 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:14.717988 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:14.717956 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:17.017115 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:17.017081 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:17.086941 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:17.086908 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:20.114052 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.113994 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-688c6dc994-z8m27" podUID="88f49595-8ba2-46ff-ba46-39d304741499" containerName="console" containerID="cri-o://767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835" gracePeriod=15 Jul 10 02:48:20.366234 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.366163 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:48:20.465718 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.465679 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc429\" (UniqueName: \"kubernetes.io/projected/88f49595-8ba2-46ff-ba46-39d304741499-kube-api-access-fc429\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.465889 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.465735 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-serving-cert\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.465889 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.465771 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-service-ca\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.465969 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.465919 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-trusted-ca-bundle\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.466026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.465965 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-oauth-config\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.466026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466019 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-oauth-serving-cert\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.466125 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466049 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-console-config\") pod \"88f49595-8ba2-46ff-ba46-39d304741499\" (UID: \"88f49595-8ba2-46ff-ba46-39d304741499\") " Jul 10 02:48:20.466260 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466219 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-service-ca" (OuterVolumeSpecName: "service-ca") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:48:20.466354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466298 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:20.466415 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466360 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:48:20.466415 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466395 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:48:20.466538 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.466518 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-console-config" (OuterVolumeSpecName: "console-config") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:48:20.468067 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.468039 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:48:20.468352 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.468331 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:48:20.468415 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.468334 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f49595-8ba2-46ff-ba46-39d304741499-kube-api-access-fc429" (OuterVolumeSpecName: "kube-api-access-fc429") pod "88f49595-8ba2-46ff-ba46-39d304741499" (UID: "88f49595-8ba2-46ff-ba46-39d304741499"). InnerVolumeSpecName "kube-api-access-fc429". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:48:20.485135 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.485105 2585 generic.go:356] "Generic (PLEG): container finished" podID="88f49595-8ba2-46ff-ba46-39d304741499" containerID="767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835" exitCode=0 Jul 10 02:48:20.485242 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.485161 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688c6dc994-z8m27" Jul 10 02:48:20.485320 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.485162 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c6dc994-z8m27" event={"ID":"88f49595-8ba2-46ff-ba46-39d304741499","Type":"ContainerDied","Data":"767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835"} Jul 10 02:48:20.485320 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.485289 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688c6dc994-z8m27" event={"ID":"88f49595-8ba2-46ff-ba46-39d304741499","Type":"ContainerDied","Data":"1ff5589497b7887aa18fd7aba832ea600a8a41b81ecedb4f96995e3e3efc1c5b"} Jul 10 02:48:20.485320 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.485317 2585 scope.go:117] "RemoveContainer" containerID="767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835" Jul 10 02:48:20.496327 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.496307 2585 scope.go:117] "RemoveContainer" containerID="767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835" Jul 10 02:48:20.496565 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:20.496548 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835\": container with ID starting with 767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835 not found: ID does not exist" containerID="767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835" Jul 10 02:48:20.496611 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.496574 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835"} err="failed to get container status \"767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835\": rpc error: code = NotFound desc = could not find container \"767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835\": container with ID starting with 767415d78cebcdb02085acfce50d74c98444fe2b7611ab3ca353d421b3bd5835 not found: ID does not exist" Jul 10 02:48:20.512559 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.512531 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688c6dc994-z8m27"] Jul 10 02:48:20.515814 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.515792 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688c6dc994-z8m27"] Jul 10 02:48:20.566980 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.566947 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-trusted-ca-bundle\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:20.566980 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.566975 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-oauth-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:20.566980 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.566986 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-oauth-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:20.567218 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.566995 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/88f49595-8ba2-46ff-ba46-39d304741499-console-config\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:20.567218 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.567004 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc429\" (UniqueName: \"kubernetes.io/projected/88f49595-8ba2-46ff-ba46-39d304741499-kube-api-access-fc429\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:20.567218 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:20.567013 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/88f49595-8ba2-46ff-ba46-39d304741499-console-serving-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:22.111023 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:22.110993 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88f49595-8ba2-46ff-ba46-39d304741499" path="/var/lib/kubelet/pods/88f49595-8ba2-46ff-ba46-39d304741499/volumes" Jul 10 02:48:22.217160 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:22.217125 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:22.317545 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:22.317507 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:23.449046 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:23.449014 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:24.454306 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.454272 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:24.507381 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.507343 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl"] Jul 10 02:48:24.507607 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.507579 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" podUID="ef270adf-55e7-4abb-aa61-3b2bb7db6e40" containerName="manager" containerID="cri-o://bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253" gracePeriod=10 Jul 10 02:48:24.724290 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.724206 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr"] Jul 10 02:48:24.724805 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.724786 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88f49595-8ba2-46ff-ba46-39d304741499" containerName="console" Jul 10 02:48:24.724848 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.724809 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="88f49595-8ba2-46ff-ba46-39d304741499" containerName="console" Jul 10 02:48:24.724921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.724912 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="88f49595-8ba2-46ff-ba46-39d304741499" containerName="console" Jul 10 02:48:24.728404 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.728380 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:24.747676 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.747624 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr"] Jul 10 02:48:24.779458 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.779433 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:24.799811 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.799779 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tht7k\" (UniqueName: \"kubernetes.io/projected/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-kube-api-access-tht7k\") pod \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " Jul 10 02:48:24.799811 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.799821 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-extensions-socket-volume\") pod \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\" (UID: \"ef270adf-55e7-4abb-aa61-3b2bb7db6e40\") " Jul 10 02:48:24.800135 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.800109 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6fnb\" (UniqueName: \"kubernetes.io/projected/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-kube-api-access-w6fnb\") pod \"kuadrant-operator-controller-manager-86878c769f-7gjvr\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:24.800234 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.800172 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7gjvr\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:24.800829 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.800634 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ef270adf-55e7-4abb-aa61-3b2bb7db6e40" (UID: "ef270adf-55e7-4abb-aa61-3b2bb7db6e40"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:48:24.802388 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.802330 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-kube-api-access-tht7k" (OuterVolumeSpecName: "kube-api-access-tht7k") pod "ef270adf-55e7-4abb-aa61-3b2bb7db6e40" (UID: "ef270adf-55e7-4abb-aa61-3b2bb7db6e40"). InnerVolumeSpecName "kube-api-access-tht7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:48:24.901253 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.901212 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6fnb\" (UniqueName: \"kubernetes.io/projected/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-kube-api-access-w6fnb\") pod \"kuadrant-operator-controller-manager-86878c769f-7gjvr\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:24.901411 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.901263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7gjvr\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:24.901411 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.901294 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tht7k\" (UniqueName: \"kubernetes.io/projected/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-kube-api-access-tht7k\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:24.901411 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.901306 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef270adf-55e7-4abb-aa61-3b2bb7db6e40-extensions-socket-volume\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:24.901625 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.901609 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7gjvr\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:24.917339 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:24.917300 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6fnb\" (UniqueName: \"kubernetes.io/projected/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-kube-api-access-w6fnb\") pod \"kuadrant-operator-controller-manager-86878c769f-7gjvr\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:25.038415 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.038320 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:25.173513 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.173489 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr"] Jul 10 02:48:25.175405 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:48:25.175375 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4ba932_5ff7_4f08_aac0_0a4afb6a9832.slice/crio-a48d3e1a999acef986cd6ec1dadf863f70b87ae6986c612e257f50cefb0e8363 WatchSource:0}: Error finding container a48d3e1a999acef986cd6ec1dadf863f70b87ae6986c612e257f50cefb0e8363: Status 404 returned error can't find the container with id a48d3e1a999acef986cd6ec1dadf863f70b87ae6986c612e257f50cefb0e8363 Jul 10 02:48:25.514565 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.514537 2585 generic.go:356] "Generic (PLEG): container finished" podID="ef270adf-55e7-4abb-aa61-3b2bb7db6e40" containerID="bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253" exitCode=0 Jul 10 02:48:25.515026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.514574 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" Jul 10 02:48:25.515026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.514617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" event={"ID":"ef270adf-55e7-4abb-aa61-3b2bb7db6e40","Type":"ContainerDied","Data":"bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253"} Jul 10 02:48:25.515026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.514687 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl" event={"ID":"ef270adf-55e7-4abb-aa61-3b2bb7db6e40","Type":"ContainerDied","Data":"655144bc6303a11237ddf530a2ec511d1413de7dcd0f3634f77e78166debf217"} Jul 10 02:48:25.515026 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.514719 2585 scope.go:117] "RemoveContainer" containerID="bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253" Jul 10 02:48:25.516408 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.516384 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" event={"ID":"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832","Type":"ContainerStarted","Data":"6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c"} Jul 10 02:48:25.516504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.516438 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" event={"ID":"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832","Type":"ContainerStarted","Data":"a48d3e1a999acef986cd6ec1dadf863f70b87ae6986c612e257f50cefb0e8363"} Jul 10 02:48:25.516504 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.516476 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:25.524232 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.524216 2585 scope.go:117] "RemoveContainer" containerID="bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253" Jul 10 02:48:25.524526 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:25.524508 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253\": container with ID starting with bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253 not found: ID does not exist" containerID="bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253" Jul 10 02:48:25.524566 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.524534 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253"} err="failed to get container status \"bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253\": rpc error: code = NotFound desc = could not find container \"bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253\": container with ID starting with bc59e6ccbdbf75273e2923da8424a869efadf29885a0237c805488830bcb4253 not found: ID does not exist" Jul 10 02:48:25.557839 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.557789 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" podStartSLOduration=1.557772333 podStartE2EDuration="1.557772333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:25.556219448 +0000 UTC m=+512.265027982" watchObservedRunningTime="2026-07-10 02:48:25.557772333 +0000 UTC m=+512.266580868" Jul 10 02:48:25.596778 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.596746 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl"] Jul 10 02:48:25.609753 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:25.609722 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rspsl"] Jul 10 02:48:26.111786 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:26.111750 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef270adf-55e7-4abb-aa61-3b2bb7db6e40" path="/var/lib/kubelet/pods/ef270adf-55e7-4abb-aa61-3b2bb7db6e40/volumes" Jul 10 02:48:28.854482 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:28.854444 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:28.955886 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:28.955849 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:36.525028 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.524999 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 02:48:36.579245 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.579209 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld"] Jul 10 02:48:36.579469 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.579446 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" podUID="741c3d57-db62-4f82-8881-3869f3f87a75" containerName="manager" containerID="cri-o://a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413" gracePeriod=10 Jul 10 02:48:36.826837 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.826812 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:36.907921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.907890 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/741c3d57-db62-4f82-8881-3869f3f87a75-extensions-socket-volume\") pod \"741c3d57-db62-4f82-8881-3869f3f87a75\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " Jul 10 02:48:36.908110 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.907956 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qjdt\" (UniqueName: \"kubernetes.io/projected/741c3d57-db62-4f82-8881-3869f3f87a75-kube-api-access-5qjdt\") pod \"741c3d57-db62-4f82-8881-3869f3f87a75\" (UID: \"741c3d57-db62-4f82-8881-3869f3f87a75\") " Jul 10 02:48:36.908232 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.908207 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/741c3d57-db62-4f82-8881-3869f3f87a75-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "741c3d57-db62-4f82-8881-3869f3f87a75" (UID: "741c3d57-db62-4f82-8881-3869f3f87a75"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 02:48:36.909948 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:36.909925 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/741c3d57-db62-4f82-8881-3869f3f87a75-kube-api-access-5qjdt" (OuterVolumeSpecName: "kube-api-access-5qjdt") pod "741c3d57-db62-4f82-8881-3869f3f87a75" (UID: "741c3d57-db62-4f82-8881-3869f3f87a75"). InnerVolumeSpecName "kube-api-access-5qjdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:48:37.008797 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.008762 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/741c3d57-db62-4f82-8881-3869f3f87a75-extensions-socket-volume\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:37.008797 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.008793 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qjdt\" (UniqueName: \"kubernetes.io/projected/741c3d57-db62-4f82-8881-3869f3f87a75-kube-api-access-5qjdt\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:48:37.583658 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.583615 2585 generic.go:356] "Generic (PLEG): container finished" podID="741c3d57-db62-4f82-8881-3869f3f87a75" containerID="a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413" exitCode=0 Jul 10 02:48:37.584108 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.583671 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" Jul 10 02:48:37.584108 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.583689 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" event={"ID":"741c3d57-db62-4f82-8881-3869f3f87a75","Type":"ContainerDied","Data":"a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413"} Jul 10 02:48:37.584108 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.583733 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld" event={"ID":"741c3d57-db62-4f82-8881-3869f3f87a75","Type":"ContainerDied","Data":"2a9b40d6099526764c2bf6ee298b9e251accf6d2b60f50b83a4906b09ad20bda"} Jul 10 02:48:37.584108 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.583754 2585 scope.go:117] "RemoveContainer" containerID="a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413" Jul 10 02:48:37.592946 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.592931 2585 scope.go:117] "RemoveContainer" containerID="a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413" Jul 10 02:48:37.593192 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:48:37.593176 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413\": container with ID starting with a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413 not found: ID does not exist" containerID="a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413" Jul 10 02:48:37.593231 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.593201 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413"} err="failed to get container status \"a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413\": rpc error: code = NotFound desc = could not find container \"a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413\": container with ID starting with a537e9f91d255ba2b7fffce60cec6fe3aa502f20ca2d954349bc3d0fa89f5413 not found: ID does not exist" Jul 10 02:48:37.606957 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.606932 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld"] Jul 10 02:48:37.612014 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:37.611990 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-p5cld"] Jul 10 02:48:38.112247 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:38.112216 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="741c3d57-db62-4f82-8881-3869f3f87a75" path="/var/lib/kubelet/pods/741c3d57-db62-4f82-8881-3869f3f87a75/volumes" Jul 10 02:48:39.329411 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:39.329381 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:41.015906 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:41.015871 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:41.265378 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:41.265343 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:57.972771 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.972734 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:48:57.973174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.973070 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef270adf-55e7-4abb-aa61-3b2bb7db6e40" containerName="manager" Jul 10 02:48:57.973174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.973080 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef270adf-55e7-4abb-aa61-3b2bb7db6e40" containerName="manager" Jul 10 02:48:57.973174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.973100 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="741c3d57-db62-4f82-8881-3869f3f87a75" containerName="manager" Jul 10 02:48:57.973174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.973106 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="741c3d57-db62-4f82-8881-3869f3f87a75" containerName="manager" Jul 10 02:48:57.973174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.973155 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="741c3d57-db62-4f82-8881-3869f3f87a75" containerName="manager" Jul 10 02:48:57.973174 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.973164 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef270adf-55e7-4abb-aa61-3b2bb7db6e40" containerName="manager" Jul 10 02:48:57.977854 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.977837 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:57.981286 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.981261 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xwllg\"" Jul 10 02:48:57.981404 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.981267 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 02:48:57.987617 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.987587 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:57.988606 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:57.988586 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:48:58.069451 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.069412 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:48:58.084871 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.084840 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:58.087939 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.087910 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47pnh\" (UniqueName: \"kubernetes.io/projected/a4d221ba-f688-4489-add4-915c6610c33b-kube-api-access-47pnh\") pod \"limitador-limitador-64c94fd64f-99t2z\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.088062 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.087968 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4d221ba-f688-4489-add4-915c6610c33b-config-file\") pod \"limitador-limitador-64c94fd64f-99t2z\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.158810 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.158778 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:58.189253 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.189221 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4d221ba-f688-4489-add4-915c6610c33b-config-file\") pod \"limitador-limitador-64c94fd64f-99t2z\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.189424 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.189309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47pnh\" (UniqueName: \"kubernetes.io/projected/a4d221ba-f688-4489-add4-915c6610c33b-kube-api-access-47pnh\") pod \"limitador-limitador-64c94fd64f-99t2z\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.189937 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.189915 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4d221ba-f688-4489-add4-915c6610c33b-config-file\") pod \"limitador-limitador-64c94fd64f-99t2z\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.198022 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.197993 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47pnh\" (UniqueName: \"kubernetes.io/projected/a4d221ba-f688-4489-add4-915c6610c33b-kube-api-access-47pnh\") pod \"limitador-limitador-64c94fd64f-99t2z\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.289381 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.289282 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:48:58.333274 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.333241 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:48:58.338279 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.338250 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.347182 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.347150 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:48:58.379909 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.379879 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:48:58.444644 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.444618 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:48:58.445989 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:48:58.445964 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d221ba_f688_4489_add4_915c6610c33b.slice/crio-f8dcfdefd15fd57957eb58f349170ed93887c25778e1d4d37a5f40ce6984bc40 WatchSource:0}: Error finding container f8dcfdefd15fd57957eb58f349170ed93887c25778e1d4d37a5f40ce6984bc40: Status 404 returned error can't find the container with id f8dcfdefd15fd57957eb58f349170ed93887c25778e1d4d37a5f40ce6984bc40 Jul 10 02:48:58.491058 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.491023 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v245\" (UniqueName: \"kubernetes.io/projected/7605e3f0-ecb3-46d9-bf33-4664f86e17f3-kube-api-access-7v245\") pod \"limitador-limitador-54d6d988d5-l2bkv\" (UID: \"7605e3f0-ecb3-46d9-bf33-4664f86e17f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.491243 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.491098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7605e3f0-ecb3-46d9-bf33-4664f86e17f3-config-file\") pod \"limitador-limitador-54d6d988d5-l2bkv\" (UID: \"7605e3f0-ecb3-46d9-bf33-4664f86e17f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.592398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.592313 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v245\" (UniqueName: \"kubernetes.io/projected/7605e3f0-ecb3-46d9-bf33-4664f86e17f3-kube-api-access-7v245\") pod \"limitador-limitador-54d6d988d5-l2bkv\" (UID: \"7605e3f0-ecb3-46d9-bf33-4664f86e17f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.592398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.592376 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7605e3f0-ecb3-46d9-bf33-4664f86e17f3-config-file\") pod \"limitador-limitador-54d6d988d5-l2bkv\" (UID: \"7605e3f0-ecb3-46d9-bf33-4664f86e17f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.592996 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.592978 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7605e3f0-ecb3-46d9-bf33-4664f86e17f3-config-file\") pod \"limitador-limitador-54d6d988d5-l2bkv\" (UID: \"7605e3f0-ecb3-46d9-bf33-4664f86e17f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.601862 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.601836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v245\" (UniqueName: \"kubernetes.io/projected/7605e3f0-ecb3-46d9-bf33-4664f86e17f3-kube-api-access-7v245\") pod \"limitador-limitador-54d6d988d5-l2bkv\" (UID: \"7605e3f0-ecb3-46d9-bf33-4664f86e17f3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.656364 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.656324 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:48:58.696399 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.696356 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" event={"ID":"a4d221ba-f688-4489-add4-915c6610c33b","Type":"ContainerStarted","Data":"f8dcfdefd15fd57957eb58f349170ed93887c25778e1d4d37a5f40ce6984bc40"} Jul 10 02:48:58.796857 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.796834 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:48:58.798437 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:48:58.798380 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7605e3f0_ecb3_46d9_bf33_4664f86e17f3.slice/crio-767fff6ec69aad055a6eb1a2c0db2a250b9a5234f07e1e458f7e50edb26eaa6a WatchSource:0}: Error finding container 767fff6ec69aad055a6eb1a2c0db2a250b9a5234f07e1e458f7e50edb26eaa6a: Status 404 returned error can't find the container with id 767fff6ec69aad055a6eb1a2c0db2a250b9a5234f07e1e458f7e50edb26eaa6a Jul 10 02:48:58.817006 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.816978 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:58.918471 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:58.918375 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:48:59.708863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:48:59.708725 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" event={"ID":"7605e3f0-ecb3-46d9-bf33-4664f86e17f3","Type":"ContainerStarted","Data":"767fff6ec69aad055a6eb1a2c0db2a250b9a5234f07e1e458f7e50edb26eaa6a"} Jul 10 02:49:01.723122 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:01.723086 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" event={"ID":"a4d221ba-f688-4489-add4-915c6610c33b","Type":"ContainerStarted","Data":"60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc"} Jul 10 02:49:01.723506 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:01.723174 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:49:01.724716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:01.724687 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" event={"ID":"7605e3f0-ecb3-46d9-bf33-4664f86e17f3","Type":"ContainerStarted","Data":"bfab8a7dd8e190431b30ef2f1fe1f12a0f5df81905fd7f8dc5a98f5afd7195d3"} Jul 10 02:49:01.724821 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:01.724743 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:49:01.744050 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:01.743985 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" podStartSLOduration=1.611647185 podStartE2EDuration="4.743970051s" totalImagesPullingTime="3.132322866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:01.741944903 +0000 UTC m=+548.450753438" watchObservedRunningTime="2026-07-10 02:49:01.743970051 +0000 UTC m=+548.452778585" Jul 10 02:49:01.761519 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:01.761459 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" podStartSLOduration=0.976973553 podStartE2EDuration="3.761439889s" totalImagesPullingTime="2.784466336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:01.760546219 +0000 UTC m=+548.469354754" watchObservedRunningTime="2026-07-10 02:49:01.761439889 +0000 UTC m=+548.470248427" Jul 10 02:49:12.730691 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:12.730635 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-l2bkv" Jul 10 02:49:12.731125 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:12.730811 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:49:12.797840 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:12.797807 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:49:12.798029 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:12.797984 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" podUID="a4d221ba-f688-4489-add4-915c6610c33b" containerName="limitador" containerID="cri-o://60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc" gracePeriod=30 Jul 10 02:49:13.349806 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.349781 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:49:13.424839 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.424755 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47pnh\" (UniqueName: \"kubernetes.io/projected/a4d221ba-f688-4489-add4-915c6610c33b-kube-api-access-47pnh\") pod \"a4d221ba-f688-4489-add4-915c6610c33b\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " Jul 10 02:49:13.424839 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.424803 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4d221ba-f688-4489-add4-915c6610c33b-config-file\") pod \"a4d221ba-f688-4489-add4-915c6610c33b\" (UID: \"a4d221ba-f688-4489-add4-915c6610c33b\") " Jul 10 02:49:13.425190 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.425165 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d221ba-f688-4489-add4-915c6610c33b-config-file" (OuterVolumeSpecName: "config-file") pod "a4d221ba-f688-4489-add4-915c6610c33b" (UID: "a4d221ba-f688-4489-add4-915c6610c33b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:49:13.426851 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.426830 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d221ba-f688-4489-add4-915c6610c33b-kube-api-access-47pnh" (OuterVolumeSpecName: "kube-api-access-47pnh") pod "a4d221ba-f688-4489-add4-915c6610c33b" (UID: "a4d221ba-f688-4489-add4-915c6610c33b"). InnerVolumeSpecName "kube-api-access-47pnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:49:13.525716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.525682 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47pnh\" (UniqueName: \"kubernetes.io/projected/a4d221ba-f688-4489-add4-915c6610c33b-kube-api-access-47pnh\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:49:13.525716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.525710 2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a4d221ba-f688-4489-add4-915c6610c33b-config-file\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:49:13.795801 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.795767 2585 generic.go:356] "Generic (PLEG): container finished" podID="a4d221ba-f688-4489-add4-915c6610c33b" containerID="60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc" exitCode=0 Jul 10 02:49:13.796196 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.795819 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" Jul 10 02:49:13.796196 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.795859 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" event={"ID":"a4d221ba-f688-4489-add4-915c6610c33b","Type":"ContainerDied","Data":"60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc"} Jul 10 02:49:13.796196 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.795916 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-99t2z" event={"ID":"a4d221ba-f688-4489-add4-915c6610c33b","Type":"ContainerDied","Data":"f8dcfdefd15fd57957eb58f349170ed93887c25778e1d4d37a5f40ce6984bc40"} Jul 10 02:49:13.796196 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.795945 2585 scope.go:117] "RemoveContainer" containerID="60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc" Jul 10 02:49:13.805414 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.805227 2585 scope.go:117] "RemoveContainer" containerID="60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc" Jul 10 02:49:13.805516 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:49:13.805498 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc\": container with ID starting with 60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc not found: ID does not exist" containerID="60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc" Jul 10 02:49:13.805562 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.805525 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc"} err="failed to get container status \"60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc\": rpc error: code = NotFound desc = could not find container \"60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc\": container with ID starting with 60554bd3ace28d28b9e5311bc1fb93d20c6d191428f25b3dda77e28d91bfd9cc not found: ID does not exist" Jul 10 02:49:13.817818 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.817796 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:49:13.821486 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:13.821464 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-99t2z"] Jul 10 02:49:14.118297 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:14.118210 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d221ba-f688-4489-add4-915c6610c33b" path="/var/lib/kubelet/pods/a4d221ba-f688-4489-add4-915c6610c33b/volumes" Jul 10 02:49:14.992149 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:14.992112 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:15.091835 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:15.091804 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:15.541433 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:15.541401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:27.854397 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:27.854360 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:28.544394 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:28.544357 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:31.467404 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:31.467348 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:34.552846 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.552801 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:34.557701 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.557672 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-754fd7f78c-zxd68"] Jul 10 02:49:34.558108 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.558091 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d221ba-f688-4489-add4-915c6610c33b" containerName="limitador" Jul 10 02:49:34.558189 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.558110 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d221ba-f688-4489-add4-915c6610c33b" containerName="limitador" Jul 10 02:49:34.558242 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.558195 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d221ba-f688-4489-add4-915c6610c33b" containerName="limitador" Jul 10 02:49:34.562908 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.562885 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.566547 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.566526 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 02:49:34.566677 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.566526 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 02:49:34.566677 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.566634 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5qr2q\"" Jul 10 02:49:34.581969 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.581934 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-754fd7f78c-zxd68"] Jul 10 02:49:34.592059 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.592028 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f092e787-2494-470c-a74f-3e4503072c10-webhook-cert\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.592220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.592066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9nv\" (UniqueName: \"kubernetes.io/projected/f092e787-2494-470c-a74f-3e4503072c10-kube-api-access-fh9nv\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.592220 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.592141 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f092e787-2494-470c-a74f-3e4503072c10-service-ca\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.653540 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.653496 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:34.692907 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.692865 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f092e787-2494-470c-a74f-3e4503072c10-webhook-cert\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.692907 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.692905 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9nv\" (UniqueName: \"kubernetes.io/projected/f092e787-2494-470c-a74f-3e4503072c10-kube-api-access-fh9nv\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.693116 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.692962 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f092e787-2494-470c-a74f-3e4503072c10-service-ca\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.693709 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.693685 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f092e787-2494-470c-a74f-3e4503072c10-service-ca\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.695354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.695325 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f092e787-2494-470c-a74f-3e4503072c10-webhook-cert\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.700963 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.700936 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9nv\" (UniqueName: \"kubernetes.io/projected/f092e787-2494-470c-a74f-3e4503072c10-kube-api-access-fh9nv\") pod \"maas-controller-754fd7f78c-zxd68\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:34.715354 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.715325 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:34.873242 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:34.873146 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:35.005216 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:35.005193 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-754fd7f78c-zxd68"] Jul 10 02:49:35.006825 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:49:35.006799 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf092e787_2494_470c_a74f_3e4503072c10.slice/crio-540b8010f650c1dabf9ff8b884854a87232ba293ac4ee6e7a14930cc5b21959f WatchSource:0}: Error finding container 540b8010f650c1dabf9ff8b884854a87232ba293ac4ee6e7a14930cc5b21959f: Status 404 returned error can't find the container with id 540b8010f650c1dabf9ff8b884854a87232ba293ac4ee6e7a14930cc5b21959f Jul 10 02:49:35.918905 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:35.918868 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754fd7f78c-zxd68" event={"ID":"f092e787-2494-470c-a74f-3e4503072c10","Type":"ContainerStarted","Data":"540b8010f650c1dabf9ff8b884854a87232ba293ac4ee6e7a14930cc5b21959f"} Jul 10 02:49:37.219288 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:37.219187 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:37.323190 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:37.323152 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:37.930523 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:37.930477 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754fd7f78c-zxd68" event={"ID":"f092e787-2494-470c-a74f-3e4503072c10","Type":"ContainerStarted","Data":"cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f"} Jul 10 02:49:37.930716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:37.930574 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:37.953491 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:37.953435 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-754fd7f78c-zxd68" podStartSLOduration=2.006020388 podStartE2EDuration="3.953418669s" totalImagesPullingTime="1.947398281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:37.950424986 +0000 UTC m=+584.659233513" watchObservedRunningTime="2026-07-10 02:49:37.953418669 +0000 UTC m=+584.662227212" Jul 10 02:49:44.565441 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:44.565398 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:49:48.941745 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:48.941712 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:49.010068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.010028 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-754fd7f78c-zxd68"] Jul 10 02:49:49.010269 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.010245 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-754fd7f78c-zxd68" podUID="f092e787-2494-470c-a74f-3e4503072c10" containerName="manager" containerID="cri-o://cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f" gracePeriod=30 Jul 10 02:49:49.255958 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.255933 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:49.335107 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.335073 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh9nv\" (UniqueName: \"kubernetes.io/projected/f092e787-2494-470c-a74f-3e4503072c10-kube-api-access-fh9nv\") pod \"f092e787-2494-470c-a74f-3e4503072c10\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " Jul 10 02:49:49.335271 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.335191 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f092e787-2494-470c-a74f-3e4503072c10-webhook-cert\") pod \"f092e787-2494-470c-a74f-3e4503072c10\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " Jul 10 02:49:49.335271 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.335234 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f092e787-2494-470c-a74f-3e4503072c10-service-ca\") pod \"f092e787-2494-470c-a74f-3e4503072c10\" (UID: \"f092e787-2494-470c-a74f-3e4503072c10\") " Jul 10 02:49:49.335665 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.335625 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f092e787-2494-470c-a74f-3e4503072c10-service-ca" (OuterVolumeSpecName: "service-ca") pod "f092e787-2494-470c-a74f-3e4503072c10" (UID: "f092e787-2494-470c-a74f-3e4503072c10"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 02:49:49.337232 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.337211 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f092e787-2494-470c-a74f-3e4503072c10-kube-api-access-fh9nv" (OuterVolumeSpecName: "kube-api-access-fh9nv") pod "f092e787-2494-470c-a74f-3e4503072c10" (UID: "f092e787-2494-470c-a74f-3e4503072c10"). InnerVolumeSpecName "kube-api-access-fh9nv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 02:49:49.337359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.337339 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f092e787-2494-470c-a74f-3e4503072c10-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f092e787-2494-470c-a74f-3e4503072c10" (UID: "f092e787-2494-470c-a74f-3e4503072c10"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 02:49:49.436398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.436360 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f092e787-2494-470c-a74f-3e4503072c10-webhook-cert\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:49:49.436398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.436388 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f092e787-2494-470c-a74f-3e4503072c10-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:49:49.436398 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.436398 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh9nv\" (UniqueName: \"kubernetes.io/projected/f092e787-2494-470c-a74f-3e4503072c10-kube-api-access-fh9nv\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 02:49:49.992693 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.992641 2585 generic.go:356] "Generic (PLEG): container finished" podID="f092e787-2494-470c-a74f-3e4503072c10" containerID="cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f" exitCode=0 Jul 10 02:49:49.993087 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.992711 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-754fd7f78c-zxd68" Jul 10 02:49:49.993087 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.992746 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754fd7f78c-zxd68" event={"ID":"f092e787-2494-470c-a74f-3e4503072c10","Type":"ContainerDied","Data":"cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f"} Jul 10 02:49:49.993087 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.992799 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-754fd7f78c-zxd68" event={"ID":"f092e787-2494-470c-a74f-3e4503072c10","Type":"ContainerDied","Data":"540b8010f650c1dabf9ff8b884854a87232ba293ac4ee6e7a14930cc5b21959f"} Jul 10 02:49:49.993087 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:49.992821 2585 scope.go:117] "RemoveContainer" containerID="cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f" Jul 10 02:49:50.002140 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:50.002120 2585 scope.go:117] "RemoveContainer" containerID="cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f" Jul 10 02:49:50.002407 ip-10-0-139-175 kubenswrapper[2585]: E0710 02:49:50.002388 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f\": container with ID starting with cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f not found: ID does not exist" containerID="cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f" Jul 10 02:49:50.002454 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:50.002417 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f"} err="failed to get container status \"cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f\": rpc error: code = NotFound desc = could not find container \"cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f\": container with ID starting with cca56ddc39804e39c08613c9f8388931a2d4a4669bb826013ea4a2ffa2a6170f not found: ID does not exist" Jul 10 02:49:50.015296 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:50.015267 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-754fd7f78c-zxd68"] Jul 10 02:49:50.019267 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:50.019243 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-754fd7f78c-zxd68"] Jul 10 02:49:50.117083 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:50.117039 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f092e787-2494-470c-a74f-3e4503072c10" path="/var/lib/kubelet/pods/f092e787-2494-470c-a74f-3e4503072c10/volumes" Jul 10 02:49:53.786710 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:53.786680 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:49:53.787168 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:53.787133 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:49:54.884448 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:49:54.884402 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:08.042989 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.042952 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:08.567725 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.567685 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq"] Jul 10 02:50:08.568167 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.568147 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f092e787-2494-470c-a74f-3e4503072c10" containerName="manager" Jul 10 02:50:08.568257 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.568168 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f092e787-2494-470c-a74f-3e4503072c10" containerName="manager" Jul 10 02:50:08.568311 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.568276 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f092e787-2494-470c-a74f-3e4503072c10" containerName="manager" Jul 10 02:50:08.570540 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.570519 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.573392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.573370 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 02:50:08.573514 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.573459 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 02:50:08.573587 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.573566 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 02:50:08.574629 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.574604 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pcjsl\"" Jul 10 02:50:08.588338 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.588313 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq"] Jul 10 02:50:08.703250 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.703210 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76x4j\" (UniqueName: \"kubernetes.io/projected/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-kube-api-access-76x4j\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.703434 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.703260 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-openshift-service-ca\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.703434 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.703343 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-maas-api-tls\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.804011 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.803972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-maas-api-tls\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.804202 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.804051 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76x4j\" (UniqueName: \"kubernetes.io/projected/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-kube-api-access-76x4j\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.804202 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.804088 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-openshift-service-ca\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.804783 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.804761 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-openshift-service-ca\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.806564 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.806541 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-maas-api-tls\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.812375 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.812354 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76x4j\" (UniqueName: \"kubernetes.io/projected/ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5-kube-api-access-76x4j\") pod \"maas-api-6b87df5b5f-zbrdq\" (UID: \"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5\") " pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:08.880771 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:08.880687 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:09.032807 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.032782 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq"] Jul 10 02:50:09.034942 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:50:09.034903 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2a51f8_8cae_4598_80c7_0dd7cf46dcb5.slice/crio-282fbc3a2ea07356134d59e0410c270514b591dce1565e72ebfbf4490adaec80 WatchSource:0}: Error finding container 282fbc3a2ea07356134d59e0410c270514b591dce1565e72ebfbf4490adaec80: Status 404 returned error can't find the container with id 282fbc3a2ea07356134d59e0410c270514b591dce1565e72ebfbf4490adaec80 Jul 10 02:50:09.036339 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.036320 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 02:50:09.091915 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.091871 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" event={"ID":"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5","Type":"ContainerStarted","Data":"282fbc3a2ea07356134d59e0410c270514b591dce1565e72ebfbf4490adaec80"} Jul 10 02:50:09.120375 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.120345 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-njkmk"] Jul 10 02:50:09.123443 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.123424 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.126015 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.125996 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 02:50:09.126121 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.126056 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-z5br7\"" Jul 10 02:50:09.133558 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.133504 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-njkmk"] Jul 10 02:50:09.137306 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.137286 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5thg8"] Jul 10 02:50:09.139602 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.139588 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.152697 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.152674 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5thg8"] Jul 10 02:50:09.207135 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.207103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dca6e4af-0ff6-4c18-904a-8b7f0307b109-plugins-config-volume\") pod \"payload-processing-58f86c7798-5thg8\" (UID: \"dca6e4af-0ff6-4c18-904a-8b7f0307b109\") " pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.207347 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.207170 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjt8p\" (UniqueName: \"kubernetes.io/projected/dca6e4af-0ff6-4c18-904a-8b7f0307b109-kube-api-access-wjt8p\") pod \"payload-processing-58f86c7798-5thg8\" (UID: \"dca6e4af-0ff6-4c18-904a-8b7f0307b109\") " pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.267090 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.267054 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:50:09.308197 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.308163 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dca6e4af-0ff6-4c18-904a-8b7f0307b109-plugins-config-volume\") pod \"payload-processing-58f86c7798-5thg8\" (UID: \"dca6e4af-0ff6-4c18-904a-8b7f0307b109\") " pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.308393 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.308224 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjt8p\" (UniqueName: \"kubernetes.io/projected/dca6e4af-0ff6-4c18-904a-8b7f0307b109-kube-api-access-wjt8p\") pod \"payload-processing-58f86c7798-5thg8\" (UID: \"dca6e4af-0ff6-4c18-904a-8b7f0307b109\") " pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.308393 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.308257 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfg96\" (UniqueName: \"kubernetes.io/projected/64d09dd8-c14c-4453-946f-8b4838c3c28f-kube-api-access-jfg96\") pod \"payload-pre-processing-7964457696-njkmk\" (UID: \"64d09dd8-c14c-4453-946f-8b4838c3c28f\") " pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.308393 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.308306 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d09dd8-c14c-4453-946f-8b4838c3c28f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-njkmk\" (UID: \"64d09dd8-c14c-4453-946f-8b4838c3c28f\") " pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.309121 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.309092 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dca6e4af-0ff6-4c18-904a-8b7f0307b109-plugins-config-volume\") pod \"payload-processing-58f86c7798-5thg8\" (UID: \"dca6e4af-0ff6-4c18-904a-8b7f0307b109\") " pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.333047 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.333016 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjt8p\" (UniqueName: \"kubernetes.io/projected/dca6e4af-0ff6-4c18-904a-8b7f0307b109-kube-api-access-wjt8p\") pod \"payload-processing-58f86c7798-5thg8\" (UID: \"dca6e4af-0ff6-4c18-904a-8b7f0307b109\") " pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.399860 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.399770 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:09.408817 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.408788 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfg96\" (UniqueName: \"kubernetes.io/projected/64d09dd8-c14c-4453-946f-8b4838c3c28f-kube-api-access-jfg96\") pod \"payload-pre-processing-7964457696-njkmk\" (UID: \"64d09dd8-c14c-4453-946f-8b4838c3c28f\") " pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.408980 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.408837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d09dd8-c14c-4453-946f-8b4838c3c28f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-njkmk\" (UID: \"64d09dd8-c14c-4453-946f-8b4838c3c28f\") " pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.409555 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.409532 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64d09dd8-c14c-4453-946f-8b4838c3c28f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-njkmk\" (UID: \"64d09dd8-c14c-4453-946f-8b4838c3c28f\") " pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.422484 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.422462 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfg96\" (UniqueName: \"kubernetes.io/projected/64d09dd8-c14c-4453-946f-8b4838c3c28f-kube-api-access-jfg96\") pod \"payload-pre-processing-7964457696-njkmk\" (UID: \"64d09dd8-c14c-4453-946f-8b4838c3c28f\") " pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.435319 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.435294 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:09.450035 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.450014 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:09.584163 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.584050 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-njkmk"] Jul 10 02:50:09.587225 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:50:09.587194 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64d09dd8_c14c_4453_946f_8b4838c3c28f.slice/crio-8005f1d411f2c7cdb01ebf171659cc7750d449aff688ad50c990ef00c9c4caea WatchSource:0}: Error finding container 8005f1d411f2c7cdb01ebf171659cc7750d449aff688ad50c990ef00c9c4caea: Status 404 returned error can't find the container with id 8005f1d411f2c7cdb01ebf171659cc7750d449aff688ad50c990ef00c9c4caea Jul 10 02:50:09.613367 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:09.613234 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5thg8"] Jul 10 02:50:09.615410 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:50:09.615383 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca6e4af_0ff6_4c18_904a_8b7f0307b109.slice/crio-12d76e231fc20abebe530af1def68a4a415084e04ba601a76dd0f50ed81cc4e6 WatchSource:0}: Error finding container 12d76e231fc20abebe530af1def68a4a415084e04ba601a76dd0f50ed81cc4e6: Status 404 returned error can't find the container with id 12d76e231fc20abebe530af1def68a4a415084e04ba601a76dd0f50ed81cc4e6 Jul 10 02:50:10.101127 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:10.101088 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" event={"ID":"64d09dd8-c14c-4453-946f-8b4838c3c28f","Type":"ContainerStarted","Data":"8005f1d411f2c7cdb01ebf171659cc7750d449aff688ad50c990ef00c9c4caea"} Jul 10 02:50:10.102481 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:10.102425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" event={"ID":"dca6e4af-0ff6-4c18-904a-8b7f0307b109","Type":"ContainerStarted","Data":"12d76e231fc20abebe530af1def68a4a415084e04ba601a76dd0f50ed81cc4e6"} Jul 10 02:50:12.121825 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.121788 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" event={"ID":"ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5","Type":"ContainerStarted","Data":"a0dc633702555e09e97178550cc8076c0ff10864e87eaa3768ca42d9a62a8af7"} Jul 10 02:50:12.122283 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.121911 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:12.123281 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.123258 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" event={"ID":"64d09dd8-c14c-4453-946f-8b4838c3c28f","Type":"ContainerStarted","Data":"0455c8bf84e464f9d261df39fa5e8106e340ce8a216e548c938d0a2a51ca3638"} Jul 10 02:50:12.123392 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.123311 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:12.124765 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.124741 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" event={"ID":"dca6e4af-0ff6-4c18-904a-8b7f0307b109","Type":"ContainerStarted","Data":"f04d61f5bc778a66a5e9598ced54e58ea808ce3f4865bb29127e268f84642ec3"} Jul 10 02:50:12.124866 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.124782 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:12.142887 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.142842 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" podStartSLOduration=2.115980371 podStartE2EDuration="4.142828345s" totalImagesPullingTime="2.026847974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:50:12.139559184 +0000 UTC m=+618.848367734" watchObservedRunningTime="2026-07-10 02:50:12.142828345 +0000 UTC m=+618.851636880" Jul 10 02:50:12.156462 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.156410 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" podStartSLOduration=1.684064145 podStartE2EDuration="3.156398349s" totalImagesPullingTime="1.472334204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:50:12.155052385 +0000 UTC m=+618.863860919" watchObservedRunningTime="2026-07-10 02:50:12.156398349 +0000 UTC m=+618.865206904" Jul 10 02:50:12.172007 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:12.171964 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" podStartSLOduration=1.7234096399999999 podStartE2EDuration="3.17195013s" totalImagesPullingTime="1.44854049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:50:12.170027965 +0000 UTC m=+618.878836502" watchObservedRunningTime="2026-07-10 02:50:12.17195013 +0000 UTC m=+618.880758643" Jul 10 02:50:14.841129 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:14.841093 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:18.135332 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:18.135261 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b87df5b5f-zbrdq" Jul 10 02:50:19.197498 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:19.197463 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:19.291552 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:19.291515 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:23.131875 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:23.131839 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5thg8" Jul 10 02:50:23.132301 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:23.132020 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-njkmk" Jul 10 02:50:40.293671 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:40.293568 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:40.827228 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:40.827194 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:50:58.951186 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.951146 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8"] Jul 10 02:50:58.955362 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.955336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:58.958691 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.958634 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 02:50:58.959812 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.959789 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4zkwr\"" Jul 10 02:50:58.959938 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.959865 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 02:50:58.959938 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.959868 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 02:50:58.966258 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.966235 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8"] Jul 10 02:50:58.993071 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:58.993047 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:50:59.151142 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.151111 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0336965-8245-43c5-84c8-1cbb27cbc1bd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.151315 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.151154 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.151315 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.151195 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.151315 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.151221 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.151315 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.151265 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.151315 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.151298 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj9v9\" (UniqueName: \"kubernetes.io/projected/c0336965-8245-43c5-84c8-1cbb27cbc1bd-kube-api-access-tj9v9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0336965-8245-43c5-84c8-1cbb27cbc1bd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252422 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252597 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252715 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252880 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252741 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj9v9\" (UniqueName: \"kubernetes.io/projected/c0336965-8245-43c5-84c8-1cbb27cbc1bd-kube-api-access-tj9v9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252936 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252873 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.252997 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.252945 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.253069 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.253043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.254660 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.254625 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0336965-8245-43c5-84c8-1cbb27cbc1bd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.254945 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.254928 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0336965-8245-43c5-84c8-1cbb27cbc1bd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.264684 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.264644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj9v9\" (UniqueName: \"kubernetes.io/projected/c0336965-8245-43c5-84c8-1cbb27cbc1bd-kube-api-access-tj9v9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-n6nh8\" (UID: \"c0336965-8245-43c5-84c8-1cbb27cbc1bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.267521 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.267502 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:50:59.401918 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.401888 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8"] Jul 10 02:50:59.403506 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:50:59.403484 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0336965_8245_43c5_84c8_1cbb27cbc1bd.slice/crio-234f3925e98dda6e5fdd3365280877a4a1d1988747e172a8d98ad7315f62fa2b WatchSource:0}: Error finding container 234f3925e98dda6e5fdd3365280877a4a1d1988747e172a8d98ad7315f62fa2b: Status 404 returned error can't find the container with id 234f3925e98dda6e5fdd3365280877a4a1d1988747e172a8d98ad7315f62fa2b Jul 10 02:50:59.650479 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:50:59.650347 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:51:00.380850 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:00.380811 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" event={"ID":"c0336965-8245-43c5-84c8-1cbb27cbc1bd","Type":"ContainerStarted","Data":"234f3925e98dda6e5fdd3365280877a4a1d1988747e172a8d98ad7315f62fa2b"} Jul 10 02:51:01.387380 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:01.387343 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" event={"ID":"c0336965-8245-43c5-84c8-1cbb27cbc1bd","Type":"ContainerStarted","Data":"4a54e5feefcc40a80eba99d10458c0bccb6da3bce443e76fd82ae7cb615c7901"} Jul 10 02:51:01.405385 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:01.405328 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" podStartSLOduration=2.358426787 podStartE2EDuration="3.405307874s" totalImagesPullingTime="1.046881087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:51:01.404722302 +0000 UTC m=+668.113530839" watchObservedRunningTime="2026-07-10 02:51:01.405307874 +0000 UTC m=+668.114116408" Jul 10 02:51:09.268285 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:09.268245 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:51:09.268771 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:09.268319 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:51:09.280263 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:09.280243 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:51:09.439697 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:09.439671 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-n6nh8" Jul 10 02:51:21.777621 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:21.777583 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:51:22.851659 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:22.851621 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:51:33.169599 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:33.169560 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:51:56.904872 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:56.904794 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:51:58.553027 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:51:58.552987 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:52:10.790623 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.790585 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv"] Jul 10 02:52:10.794170 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.794147 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:10.797028 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.797004 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 02:52:10.811872 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.811838 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv"] Jul 10 02:52:10.841068 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.841033 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:52:10.900481 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.900447 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a441f833-ad77-43a4-943a-45406d9b7824-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:10.900694 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.900489 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:10.900694 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.900510 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:10.900694 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.900595 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:10.900694 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.900674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:10.900860 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:10.900692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbhlh\" (UniqueName: \"kubernetes.io/projected/a441f833-ad77-43a4-943a-45406d9b7824-kube-api-access-kbhlh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002205 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002175 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002205 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002281 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002307 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbhlh\" (UniqueName: \"kubernetes.io/projected/a441f833-ad77-43a4-943a-45406d9b7824-kube-api-access-kbhlh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002464 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002396 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a441f833-ad77-43a4-943a-45406d9b7824-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002794 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002794 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002706 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.002976 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.002954 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.005304 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.005280 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a441f833-ad77-43a4-943a-45406d9b7824-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.005804 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.005570 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a441f833-ad77-43a4-943a-45406d9b7824-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.013438 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.013405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbhlh\" (UniqueName: \"kubernetes.io/projected/a441f833-ad77-43a4-943a-45406d9b7824-kube-api-access-kbhlh\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v5txv\" (UID: \"a441f833-ad77-43a4-943a-45406d9b7824\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.105510 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.105423 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:11.248497 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.248469 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv"] Jul 10 02:52:11.250137 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:52:11.250108 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda441f833_ad77_43a4_943a_45406d9b7824.slice/crio-cdc7a551ac682536a67ee78fef9524abb827b31499772dbc7c9d63142998cf04 WatchSource:0}: Error finding container cdc7a551ac682536a67ee78fef9524abb827b31499772dbc7c9d63142998cf04: Status 404 returned error can't find the container with id cdc7a551ac682536a67ee78fef9524abb827b31499772dbc7c9d63142998cf04 Jul 10 02:52:11.757381 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.757343 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" event={"ID":"a441f833-ad77-43a4-943a-45406d9b7824","Type":"ContainerStarted","Data":"863a17d344918cc1f72bfaab39678be080d62c9a42cb0911ffe8605f9237c493"} Jul 10 02:52:11.757556 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.757396 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" event={"ID":"a441f833-ad77-43a4-943a-45406d9b7824","Type":"ContainerStarted","Data":"cdc7a551ac682536a67ee78fef9524abb827b31499772dbc7c9d63142998cf04"} Jul 10 02:52:11.776440 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:11.776384 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" podStartSLOduration=1.607433457 podStartE2EDuration="1.776363476s" totalImagesPullingTime="168.930019ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:52:11.776223881 +0000 UTC m=+738.485032415" watchObservedRunningTime="2026-07-10 02:52:11.776363476 +0000 UTC m=+738.485172014" Jul 10 02:52:12.861270 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:12.861235 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:52:21.105928 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:21.105891 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:21.106302 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:21.105960 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:21.118286 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:21.118263 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:21.822595 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:21.822565 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v5txv" Jul 10 02:52:24.786221 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:24.786183 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:52:27.356437 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:52:27.356404 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:53:37.011609 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:53:37.011576 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:53:37.111805 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:53:37.111765 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:53:38.269840 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:53:38.269800 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:53:38.368984 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:53:38.368957 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:54:00.656405 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:00.656364 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:54:00.832016 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:00.831983 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:54:07.727004 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:07.726960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:54:09.750276 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:09.750238 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:54:23.848593 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:23.848553 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:54:32.849613 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:32.849572 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:54:41.152503 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:41.152450 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:54:51.654863 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:51.654780 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:54:53.830716 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:53.830685 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:54:53.831223 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:53.831201 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:54:59.650921 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:54:59.650878 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:55:01.346246 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:01.346208 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:55:03.014298 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.014268 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:55:03.136260 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.136228 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f9ff5cc88-wlwgw"] Jul 10 02:55:03.140010 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.139989 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.143774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.143743 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 02:55:03.143897 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.143805 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5qr2q\"" Jul 10 02:55:03.143897 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.143749 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 02:55:03.148135 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.148115 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f9ff5cc88-wlwgw"] Jul 10 02:55:03.158821 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.158793 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d1a423b-56b2-41d2-95f0-ab20d745c594-service-ca\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.158947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.158832 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d1a423b-56b2-41d2-95f0-ab20d745c594-webhook-cert\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.158947 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.158892 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99qp4\" (UniqueName: \"kubernetes.io/projected/8d1a423b-56b2-41d2-95f0-ab20d745c594-kube-api-access-99qp4\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.259979 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.259938 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d1a423b-56b2-41d2-95f0-ab20d745c594-service-ca\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.260181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.259991 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d1a423b-56b2-41d2-95f0-ab20d745c594-webhook-cert\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.260181 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.260044 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99qp4\" (UniqueName: \"kubernetes.io/projected/8d1a423b-56b2-41d2-95f0-ab20d745c594-kube-api-access-99qp4\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.260627 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.260600 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d1a423b-56b2-41d2-95f0-ab20d745c594-service-ca\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.262433 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.262410 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d1a423b-56b2-41d2-95f0-ab20d745c594-webhook-cert\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.269359 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.269306 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99qp4\" (UniqueName: \"kubernetes.io/projected/8d1a423b-56b2-41d2-95f0-ab20d745c594-kube-api-access-99qp4\") pod \"maas-controller-7f9ff5cc88-wlwgw\" (UID: \"8d1a423b-56b2-41d2-95f0-ab20d745c594\") " pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.450746 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.450706 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:03.590000 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.589974 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f9ff5cc88-wlwgw"] Jul 10 02:55:03.591483 ip-10-0-139-175 kubenswrapper[2585]: W0710 02:55:03.591457 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1a423b_56b2_41d2_95f0_ab20d745c594.slice/crio-4ba239b53bbd1411d752f00880de64176b7f8329ad3daed6d24f65f49007c6d4 WatchSource:0}: Error finding container 4ba239b53bbd1411d752f00880de64176b7f8329ad3daed6d24f65f49007c6d4: Status 404 returned error can't find the container with id 4ba239b53bbd1411d752f00880de64176b7f8329ad3daed6d24f65f49007c6d4 Jul 10 02:55:03.690486 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:03.690446 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" event={"ID":"8d1a423b-56b2-41d2-95f0-ab20d745c594","Type":"ContainerStarted","Data":"4ba239b53bbd1411d752f00880de64176b7f8329ad3daed6d24f65f49007c6d4"} Jul 10 02:55:04.698309 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:04.698271 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" event={"ID":"8d1a423b-56b2-41d2-95f0-ab20d745c594","Type":"ContainerStarted","Data":"fe0218bb60a1081cb94465cc05a6ded0b62a164b98a065e9772ef1b9fa0261af"} Jul 10 02:55:04.698808 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:04.698358 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:04.743548 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:04.743500 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" podStartSLOduration=1.2240993279999999 podStartE2EDuration="1.743485287s" totalImagesPullingTime="519.385959ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:55:04.743119472 +0000 UTC m=+911.451928003" watchObservedRunningTime="2026-07-10 02:55:04.743485287 +0000 UTC m=+911.452293822" Jul 10 02:55:15.713175 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:15.713138 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f9ff5cc88-wlwgw" Jul 10 02:55:26.756208 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:26.756175 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:55:39.752845 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:39.752805 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:55:48.750846 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:48.750807 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:55:58.143493 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:58.143455 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:55:58.266316 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:58.266255 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:55:59.548127 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:55:59.548081 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:04.287458 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:04.287431 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:04.391409 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:04.391373 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:04.497555 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:04.497527 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:04.699428 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:04.699393 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:08.849543 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:08.849509 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:56:10.209539 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:10.209509 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:10.309081 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:10.309038 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:56:31.047561 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:31.047523 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:56:42.646334 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:42.646298 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:56:52.351842 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:56:52.351808 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:57:02.853964 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:02.853923 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:57:15.057889 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:15.057852 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:57:26.953805 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:26.953763 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:57:38.681665 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:38.681626 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:57:41.153262 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:41.153197 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:57:49.356161 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:49.356069 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:57:58.651872 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:57:58.651836 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:58:03.553774 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:58:03.553737 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:58:25.248376 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:58:25.248328 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:58:34.655546 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:58:34.655510 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:58:55.565432 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:58:55.565396 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:59:10.572690 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:10.572635 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:59:29.688795 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:29.688757 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 02:59:31.758038 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:31.758001 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:59:43.608326 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:43.608289 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 02:59:53.868094 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:53.868063 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:59:53.870083 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:53.870061 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 02:59:54.656404 ip-10-0-139-175 kubenswrapper[2585]: I0710 02:59:54.656363 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:00:04.288578 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:00:04.288535 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:00:14.354930 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:00:14.354889 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:00:32.261701 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:00:32.261664 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:00:43.356786 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:00:43.356739 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:00:57.951746 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:00:57.951661 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:01:18.558319 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:18.558277 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:01:21.407178 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.407138 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr"] Jul 10 03:01:21.407671 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.407392 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" podUID="0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" containerName="manager" containerID="cri-o://6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c" gracePeriod=10 Jul 10 03:01:21.672020 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.671958 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 03:01:21.736051 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.736020 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-extensions-socket-volume\") pod \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " Jul 10 03:01:21.736258 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.736075 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6fnb\" (UniqueName: \"kubernetes.io/projected/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-kube-api-access-w6fnb\") pod \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\" (UID: \"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832\") " Jul 10 03:01:21.736501 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.736468 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" (UID: "0a4ba932-5ff7-4f08-aac0-0a4afb6a9832"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 03:01:21.738277 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.738246 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-kube-api-access-w6fnb" (OuterVolumeSpecName: "kube-api-access-w6fnb") pod "0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" (UID: "0a4ba932-5ff7-4f08-aac0-0a4afb6a9832"). InnerVolumeSpecName "kube-api-access-w6fnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:01:21.809592 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.809559 2585 generic.go:356] "Generic (PLEG): container finished" podID="0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" containerID="6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c" exitCode=0 Jul 10 03:01:21.809774 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.809607 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" Jul 10 03:01:21.809774 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.809616 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" event={"ID":"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832","Type":"ContainerDied","Data":"6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c"} Jul 10 03:01:21.809774 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.809677 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr" event={"ID":"0a4ba932-5ff7-4f08-aac0-0a4afb6a9832","Type":"ContainerDied","Data":"a48d3e1a999acef986cd6ec1dadf863f70b87ae6986c612e257f50cefb0e8363"} Jul 10 03:01:21.809774 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.809699 2585 scope.go:117] "RemoveContainer" containerID="6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c" Jul 10 03:01:21.820518 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.820498 2585 scope.go:117] "RemoveContainer" containerID="6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c" Jul 10 03:01:21.820831 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:01:21.820814 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c\": container with ID starting with 6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c not found: ID does not exist" containerID="6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c" Jul 10 03:01:21.820905 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.820839 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c"} err="failed to get container status \"6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c\": rpc error: code = NotFound desc = could not find container \"6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c\": container with ID starting with 6f384f73c6733e4e7c5103a393a4d24083da9481f2492b2d9e5bc99463bf3f8c not found: ID does not exist" Jul 10 03:01:21.834017 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.833987 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr"] Jul 10 03:01:21.837565 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.837543 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-extensions-socket-volume\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:01:21.837565 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.837564 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6fnb\" (UniqueName: \"kubernetes.io/projected/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832-kube-api-access-w6fnb\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:01:21.838725 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:21.838706 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gjvr"] Jul 10 03:01:22.116375 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:01:22.116339 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" path="/var/lib/kubelet/pods/0a4ba932-5ff7-4f08-aac0-0a4afb6a9832/volumes" Jul 10 03:02:22.630691 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.630636 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm"] Jul 10 03:02:22.631295 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.631273 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" containerName="manager" Jul 10 03:02:22.631378 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.631299 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" containerName="manager" Jul 10 03:02:22.631432 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.631405 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4ba932-5ff7-4f08-aac0-0a4afb6a9832" containerName="manager" Jul 10 03:02:22.634559 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.634528 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.637858 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.637834 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4l7tn\"" Jul 10 03:02:22.653918 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.653892 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm"] Jul 10 03:02:22.794056 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.794017 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ba19253-d03c-45d7-87b1-e3fecdfc3ad5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-74gvm\" (UID: \"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.794242 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.794073 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5szz\" (UniqueName: \"kubernetes.io/projected/2ba19253-d03c-45d7-87b1-e3fecdfc3ad5-kube-api-access-l5szz\") pod \"kuadrant-operator-controller-manager-86878c769f-74gvm\" (UID: \"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.894728 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.894604 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ba19253-d03c-45d7-87b1-e3fecdfc3ad5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-74gvm\" (UID: \"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.894728 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.894699 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5szz\" (UniqueName: \"kubernetes.io/projected/2ba19253-d03c-45d7-87b1-e3fecdfc3ad5-kube-api-access-l5szz\") pod \"kuadrant-operator-controller-manager-86878c769f-74gvm\" (UID: \"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.895080 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.895055 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ba19253-d03c-45d7-87b1-e3fecdfc3ad5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-74gvm\" (UID: \"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.903760 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.903729 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5szz\" (UniqueName: \"kubernetes.io/projected/2ba19253-d03c-45d7-87b1-e3fecdfc3ad5-kube-api-access-l5szz\") pod \"kuadrant-operator-controller-manager-86878c769f-74gvm\" (UID: \"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:22.944771 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:22.944731 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:23.094348 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:23.094322 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm"] Jul 10 03:02:23.096424 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:02:23.096393 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba19253_d03c_45d7_87b1_e3fecdfc3ad5.slice/crio-54c5ae24ee8ab978c16c9563f353ab18fbb29a7d6d948d1dcd5a46cd34e188f4 WatchSource:0}: Error finding container 54c5ae24ee8ab978c16c9563f353ab18fbb29a7d6d948d1dcd5a46cd34e188f4: Status 404 returned error can't find the container with id 54c5ae24ee8ab978c16c9563f353ab18fbb29a7d6d948d1dcd5a46cd34e188f4 Jul 10 03:02:23.159909 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:23.159869 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" event={"ID":"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5","Type":"ContainerStarted","Data":"54c5ae24ee8ab978c16c9563f353ab18fbb29a7d6d948d1dcd5a46cd34e188f4"} Jul 10 03:02:24.167434 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:24.167394 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" event={"ID":"2ba19253-d03c-45d7-87b1-e3fecdfc3ad5","Type":"ContainerStarted","Data":"fd71be9df071d0c101503f9b9e8a9d0e6dab192f7ff8f0b7da54d0bec21fbc12"} Jul 10 03:02:24.167879 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:24.167473 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:24.192106 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:24.192044 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" podStartSLOduration=2.19202322 podStartE2EDuration="2.19202322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:02:24.187844618 +0000 UTC m=+1350.896653136" watchObservedRunningTime="2026-07-10 03:02:24.19202322 +0000 UTC m=+1350.900831756" Jul 10 03:02:24.328459 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:24.328420 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:02:25.903399 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:25.903343 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:02:35.176097 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:35.176060 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-74gvm" Jul 10 03:02:50.254724 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:50.254689 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:02:53.708588 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:02:53.708520 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:03:00.060790 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:03:00.060754 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:03:09.349432 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:03:09.349391 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:03:21.659442 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:03:21.659400 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:03:32.354631 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:03:32.354593 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:03:42.855217 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:03:42.855180 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:03:57.147600 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:03:57.147504 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:04:06.460795 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:06.460756 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:04:14.850123 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:14.850075 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:04:24.660412 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:24.660377 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:04:33.350900 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:33.350856 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:04:43.554807 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:43.554766 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:04:53.907501 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:53.907472 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:04:53.909796 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:53.909771 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:04:54.956989 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:04:54.956949 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:05:05.757877 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:05:05.757838 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:05:20.661865 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:05:20.661824 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:05:32.550723 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:05:32.550687 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:05:46.758021 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:05:46.757983 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:05:56.050623 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:05:56.050585 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:06:04.157094 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:06:04.157060 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:06:16.859707 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:06:16.859666 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:06:25.150990 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:06:25.150955 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:06:40.158300 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:06:40.158262 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:06:52.854194 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:06:52.854115 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:07:00.565463 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:07:00.565417 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:07:07.497765 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:07:07.497728 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:07:10.950372 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:07:10.950332 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:07:29.751271 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:07:29.751233 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:07:42.356969 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:07:42.356930 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:07:54.352605 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:07:54.352572 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:08:07.156312 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:08:07.156276 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:08:16.852392 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:08:16.852304 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:08:22.957554 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:08:22.957518 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:08:32.263377 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:08:32.263339 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:08:46.049929 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:08:46.049894 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:08:55.155034 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:08:55.154994 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:09:06.459591 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:06.459555 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:09:15.757394 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:15.757352 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:09:19.101068 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:19.101033 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:09:20.517134 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:20.517097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:09:29.261912 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:29.261864 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:09:40.250334 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:40.250290 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:09:50.145517 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:50.145430 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:09:53.234891 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:53.234853 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:09:53.944994 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:53.944968 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:09:53.949069 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:53.949047 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:09:59.661865 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:09:59.661819 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:10:05.106075 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:05.106031 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:09.865408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:09.865366 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:10:13.390711 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:13.390679 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:18.552968 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:18.552924 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:10:21.513391 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:21.513355 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:24.620959 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:24.620922 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:24.723177 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:24.723146 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:25.129634 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:25.129600 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:30.931134 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:30.931096 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:31.031880 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:31.031843 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:31.155619 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:31.155589 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:31.314589 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:31.314508 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:31.450657 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:31.450623 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:10:32.320581 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.320539 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:32.393842 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.393808 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2"] Jul 10 03:10:32.396397 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.396367 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:32.398094 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.398055 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.400559 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.400526 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a9b31aac-serving-cert\"" Jul 10 03:10:32.407505 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.407373 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2"] Jul 10 03:10:32.464578 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.464537 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:32.508506 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.508473 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08a69559-a3d7-4543-b3c5-a35b7619b388-maas-api-tls\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.508752 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.508519 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a69559-a3d7-4543-b3c5-a35b7619b388-openshift-service-ca\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.508752 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.508625 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbzhn\" (UniqueName: \"kubernetes.io/projected/08a69559-a3d7-4543-b3c5-a35b7619b388-kube-api-access-vbzhn\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.564485 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.564380 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:32.609327 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.609241 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a69559-a3d7-4543-b3c5-a35b7619b388-openshift-service-ca\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.609327 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.609294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbzhn\" (UniqueName: \"kubernetes.io/projected/08a69559-a3d7-4543-b3c5-a35b7619b388-kube-api-access-vbzhn\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.609514 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.609364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08a69559-a3d7-4543-b3c5-a35b7619b388-maas-api-tls\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.610033 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.610008 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a69559-a3d7-4543-b3c5-a35b7619b388-openshift-service-ca\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.611762 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.611738 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08a69559-a3d7-4543-b3c5-a35b7619b388-maas-api-tls\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.617743 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.617719 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbzhn\" (UniqueName: \"kubernetes.io/projected/08a69559-a3d7-4543-b3c5-a35b7619b388-kube-api-access-vbzhn\") pod \"maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.711240 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.711204 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:32.854635 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.854603 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2"] Jul 10 03:10:32.857059 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:10:32.857001 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a69559_a3d7_4543_b3c5_a35b7619b388.slice/crio-0bcb68aaa58ee405844d0ac5e9c0d78015c097685f68b0d2c3e50313637093a8 WatchSource:0}: Error finding container 0bcb68aaa58ee405844d0ac5e9c0d78015c097685f68b0d2c3e50313637093a8: Status 404 returned error can't find the container with id 0bcb68aaa58ee405844d0ac5e9c0d78015c097685f68b0d2c3e50313637093a8 Jul 10 03:10:32.859060 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.859036 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 03:10:32.934343 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.934300 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" event={"ID":"08a69559-a3d7-4543-b3c5-a35b7619b388","Type":"ContainerStarted","Data":"0bcb68aaa58ee405844d0ac5e9c0d78015c097685f68b0d2c3e50313637093a8"} Jul 10 03:10:32.983978 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:32.983939 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:33.084117 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:33.084085 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:33.941096 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:33.941062 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" event={"ID":"08a69559-a3d7-4543-b3c5-a35b7619b388","Type":"ContainerStarted","Data":"e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b"} Jul 10 03:10:33.941530 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:33.941229 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:33.958861 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:33.958808 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" podStartSLOduration=1.427691283 podStartE2EDuration="1.958790244s" totalImagesPullingTime="531.098961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:10:33.95688512 +0000 UTC m=+1840.665693649" watchObservedRunningTime="2026-07-10 03:10:33.958790244 +0000 UTC m=+1840.667598778" Jul 10 03:10:34.720439 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:34.720408 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:10:39.954341 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:39.954310 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:10:42.045395 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:10:42.045358 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:11:26.161394 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:26.161360 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:37.600860 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:37.600820 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:37.702201 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:37.702167 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:40.452482 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:40.452448 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:11:42.935342 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:42.935301 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2"] Jul 10 03:11:42.937282 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:42.935547 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" podUID="08a69559-a3d7-4543-b3c5-a35b7619b388" containerName="maas-api" containerID="cri-o://e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b" gracePeriod=30 Jul 10 03:11:42.971077 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:42.971046 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:43.188608 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.188583 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:11:43.267404 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.267365 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08a69559-a3d7-4543-b3c5-a35b7619b388-maas-api-tls\") pod \"08a69559-a3d7-4543-b3c5-a35b7619b388\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " Jul 10 03:11:43.267404 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.267411 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbzhn\" (UniqueName: \"kubernetes.io/projected/08a69559-a3d7-4543-b3c5-a35b7619b388-kube-api-access-vbzhn\") pod \"08a69559-a3d7-4543-b3c5-a35b7619b388\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " Jul 10 03:11:43.267626 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.267466 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a69559-a3d7-4543-b3c5-a35b7619b388-openshift-service-ca\") pod \"08a69559-a3d7-4543-b3c5-a35b7619b388\" (UID: \"08a69559-a3d7-4543-b3c5-a35b7619b388\") " Jul 10 03:11:43.267937 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.267905 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08a69559-a3d7-4543-b3c5-a35b7619b388-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "08a69559-a3d7-4543-b3c5-a35b7619b388" (UID: "08a69559-a3d7-4543-b3c5-a35b7619b388"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:11:43.269759 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.269725 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a69559-a3d7-4543-b3c5-a35b7619b388-kube-api-access-vbzhn" (OuterVolumeSpecName: "kube-api-access-vbzhn") pod "08a69559-a3d7-4543-b3c5-a35b7619b388" (UID: "08a69559-a3d7-4543-b3c5-a35b7619b388"). InnerVolumeSpecName "kube-api-access-vbzhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:11:43.269759 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.269749 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08a69559-a3d7-4543-b3c5-a35b7619b388-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "08a69559-a3d7-4543-b3c5-a35b7619b388" (UID: "08a69559-a3d7-4543-b3c5-a35b7619b388"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:11:43.343074 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.343038 2585 generic.go:356] "Generic (PLEG): container finished" podID="08a69559-a3d7-4543-b3c5-a35b7619b388" containerID="e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b" exitCode=0 Jul 10 03:11:43.343291 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.343097 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" Jul 10 03:11:43.343291 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.343131 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" event={"ID":"08a69559-a3d7-4543-b3c5-a35b7619b388","Type":"ContainerDied","Data":"e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b"} Jul 10 03:11:43.343291 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.343198 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2" event={"ID":"08a69559-a3d7-4543-b3c5-a35b7619b388","Type":"ContainerDied","Data":"0bcb68aaa58ee405844d0ac5e9c0d78015c097685f68b0d2c3e50313637093a8"} Jul 10 03:11:43.343291 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.343228 2585 scope.go:117] "RemoveContainer" containerID="e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b" Jul 10 03:11:43.354000 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.353974 2585 scope.go:117] "RemoveContainer" containerID="e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b" Jul 10 03:11:43.354310 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:11:43.354293 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b\": container with ID starting with e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b not found: ID does not exist" containerID="e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b" Jul 10 03:11:43.354364 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.354320 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b"} err="failed to get container status \"e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b\": rpc error: code = NotFound desc = could not find container \"e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b\": container with ID starting with e78e126bebfcb33368f8aa5a10809b92fc779ebf5eac7300cfd0e79869250b8b not found: ID does not exist" Jul 10 03:11:43.366989 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.366954 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2"] Jul 10 03:11:43.368415 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.368387 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08a69559-a3d7-4543-b3c5-a35b7619b388-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:11:43.368415 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.368416 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbzhn\" (UniqueName: \"kubernetes.io/projected/08a69559-a3d7-4543-b3c5-a35b7619b388-kube-api-access-vbzhn\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:11:43.368415 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.368432 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a69559-a3d7-4543-b3c5-a35b7619b388-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:11:43.371871 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.371799 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9b31aac-55895857c5-pjrj2"] Jul 10 03:11:43.404380 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.404348 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:43.941424 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:43.941396 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:44.114359 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:44.114324 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08a69559-a3d7-4543-b3c5-a35b7619b388" path="/var/lib/kubelet/pods/08a69559-a3d7-4543-b3c5-a35b7619b388/volumes" Jul 10 03:11:49.249981 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:49.249943 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:49.321128 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:49.321097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:49.504205 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:49.504115 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:49.568531 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:49.568493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:50.700302 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.700252 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:50.777992 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.777950 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6"] Jul 10 03:11:50.778454 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.778434 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a69559-a3d7-4543-b3c5-a35b7619b388" containerName="maas-api" Jul 10 03:11:50.778569 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.778456 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a69559-a3d7-4543-b3c5-a35b7619b388" containerName="maas-api" Jul 10 03:11:50.778634 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.778572 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a69559-a3d7-4543-b3c5-a35b7619b388" containerName="maas-api" Jul 10 03:11:50.779761 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.779734 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:50.783221 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.783202 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:50.786012 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.785982 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c9655e49-serving-cert\"" Jul 10 03:11:50.793781 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.793753 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6"] Jul 10 03:11:50.853751 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.853722 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:50.933910 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.933866 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8242ca35-77eb-4e1c-bce7-5cce8285e228-openshift-service-ca\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:50.933910 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.933907 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drg7z\" (UniqueName: \"kubernetes.io/projected/8242ca35-77eb-4e1c-bce7-5cce8285e228-kube-api-access-drg7z\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:50.934189 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.934087 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8242ca35-77eb-4e1c-bce7-5cce8285e228-maas-api-tls\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:50.954717 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:50.954619 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:51.035037 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.035002 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8242ca35-77eb-4e1c-bce7-5cce8285e228-openshift-service-ca\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.035037 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.035042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drg7z\" (UniqueName: \"kubernetes.io/projected/8242ca35-77eb-4e1c-bce7-5cce8285e228-kube-api-access-drg7z\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.035317 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.035113 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8242ca35-77eb-4e1c-bce7-5cce8285e228-maas-api-tls\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.035783 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.035755 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8242ca35-77eb-4e1c-bce7-5cce8285e228-openshift-service-ca\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.037723 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.037702 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8242ca35-77eb-4e1c-bce7-5cce8285e228-maas-api-tls\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.043998 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.043972 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drg7z\" (UniqueName: \"kubernetes.io/projected/8242ca35-77eb-4e1c-bce7-5cce8285e228-kube-api-access-drg7z\") pod \"maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.095802 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.095751 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:51.375957 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.375856 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:51.457450 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.457414 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6"] Jul 10 03:11:51.458143 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:51.458113 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:51.459166 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:11:51.459134 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8242ca35_77eb_4e1c_bce7_5cce8285e228.slice/crio-1583c54c870d213d00c9ce7578a819a4bf3584b5bb170095983cd4f378d6cf0a WatchSource:0}: Error finding container 1583c54c870d213d00c9ce7578a819a4bf3584b5bb170095983cd4f378d6cf0a: Status 404 returned error can't find the container with id 1583c54c870d213d00c9ce7578a819a4bf3584b5bb170095983cd4f378d6cf0a Jul 10 03:11:52.405105 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:52.404999 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" event={"ID":"8242ca35-77eb-4e1c-bce7-5cce8285e228","Type":"ContainerStarted","Data":"bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b"} Jul 10 03:11:52.405105 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:52.405078 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" event={"ID":"8242ca35-77eb-4e1c-bce7-5cce8285e228","Type":"ContainerStarted","Data":"1583c54c870d213d00c9ce7578a819a4bf3584b5bb170095983cd4f378d6cf0a"} Jul 10 03:11:52.405623 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:52.405193 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:11:52.432588 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:52.432531 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" podStartSLOduration=1.9342702090000001 podStartE2EDuration="2.432514844s" totalImagesPullingTime="498.244635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:11:52.430692321 +0000 UTC m=+1919.139500859" watchObservedRunningTime="2026-07-10 03:11:52.432514844 +0000 UTC m=+1919.141323379" Jul 10 03:11:52.686366 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:52.686336 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:11:53.856380 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:53.856341 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:11:58.417824 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:11:58.417791 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:13:35.874054 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:13:35.874017 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:13:56.103400 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:13:56.103365 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:13:56.222379 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:13:56.222347 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:13:59.158163 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:13:59.158123 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:14:01.579544 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.579501 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6"] Jul 10 03:14:01.580070 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.579767 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" podUID="8242ca35-77eb-4e1c-bce7-5cce8285e228" containerName="maas-api" containerID="cri-o://bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b" gracePeriod=30 Jul 10 03:14:01.605762 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.605740 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:01.830383 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.830358 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:14:01.966501 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.966417 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drg7z\" (UniqueName: \"kubernetes.io/projected/8242ca35-77eb-4e1c-bce7-5cce8285e228-kube-api-access-drg7z\") pod \"8242ca35-77eb-4e1c-bce7-5cce8285e228\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " Jul 10 03:14:01.966665 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.966518 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8242ca35-77eb-4e1c-bce7-5cce8285e228-openshift-service-ca\") pod \"8242ca35-77eb-4e1c-bce7-5cce8285e228\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " Jul 10 03:14:01.966665 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.966538 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8242ca35-77eb-4e1c-bce7-5cce8285e228-maas-api-tls\") pod \"8242ca35-77eb-4e1c-bce7-5cce8285e228\" (UID: \"8242ca35-77eb-4e1c-bce7-5cce8285e228\") " Jul 10 03:14:01.966979 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.966951 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8242ca35-77eb-4e1c-bce7-5cce8285e228-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8242ca35-77eb-4e1c-bce7-5cce8285e228" (UID: "8242ca35-77eb-4e1c-bce7-5cce8285e228"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:14:01.968596 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.968570 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8242ca35-77eb-4e1c-bce7-5cce8285e228-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8242ca35-77eb-4e1c-bce7-5cce8285e228" (UID: "8242ca35-77eb-4e1c-bce7-5cce8285e228"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:14:01.968721 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:01.968573 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8242ca35-77eb-4e1c-bce7-5cce8285e228-kube-api-access-drg7z" (OuterVolumeSpecName: "kube-api-access-drg7z") pod "8242ca35-77eb-4e1c-bce7-5cce8285e228" (UID: "8242ca35-77eb-4e1c-bce7-5cce8285e228"). InnerVolumeSpecName "kube-api-access-drg7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:14:02.067421 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.067385 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drg7z\" (UniqueName: \"kubernetes.io/projected/8242ca35-77eb-4e1c-bce7-5cce8285e228-kube-api-access-drg7z\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:14:02.067421 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.067414 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8242ca35-77eb-4e1c-bce7-5cce8285e228-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:14:02.067421 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.067424 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8242ca35-77eb-4e1c-bce7-5cce8285e228-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:14:02.143681 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.143634 2585 generic.go:356] "Generic (PLEG): container finished" podID="8242ca35-77eb-4e1c-bce7-5cce8285e228" containerID="bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b" exitCode=0 Jul 10 03:14:02.143824 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.143693 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" event={"ID":"8242ca35-77eb-4e1c-bce7-5cce8285e228","Type":"ContainerDied","Data":"bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b"} Jul 10 03:14:02.143824 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.143710 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" Jul 10 03:14:02.143824 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.143757 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6" event={"ID":"8242ca35-77eb-4e1c-bce7-5cce8285e228","Type":"ContainerDied","Data":"1583c54c870d213d00c9ce7578a819a4bf3584b5bb170095983cd4f378d6cf0a"} Jul 10 03:14:02.143824 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.143794 2585 scope.go:117] "RemoveContainer" containerID="bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b" Jul 10 03:14:02.153999 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.153979 2585 scope.go:117] "RemoveContainer" containerID="bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b" Jul 10 03:14:02.154324 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:14:02.154302 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b\": container with ID starting with bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b not found: ID does not exist" containerID="bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b" Jul 10 03:14:02.154394 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.154332 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b"} err="failed to get container status \"bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b\": rpc error: code = NotFound desc = could not find container \"bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b\": container with ID starting with bef7ca63c899a59e54f355d4cab93de45a2539c30e96474fd5858c94e7ec2d9b not found: ID does not exist" Jul 10 03:14:02.164622 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.164599 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6"] Jul 10 03:14:02.170075 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.170052 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c9655e49-5fb6fcd6f4-jjxq6"] Jul 10 03:14:02.195457 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:02.195438 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:04.112156 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:04.112125 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8242ca35-77eb-4e1c-bce7-5cce8285e228" path="/var/lib/kubelet/pods/8242ca35-77eb-4e1c-bce7-5cce8285e228/volumes" Jul 10 03:14:07.754856 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:07.754819 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:07.855001 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:07.854966 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:08.039941 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:08.039861 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:12.504776 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:12.504725 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:12.605179 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:12.605140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:15.890962 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:15.890925 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:17.174936 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.174857 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:17.261429 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.261398 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv"] Jul 10 03:14:17.261817 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.261799 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8242ca35-77eb-4e1c-bce7-5cce8285e228" containerName="maas-api" Jul 10 03:14:17.261817 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.261814 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8242ca35-77eb-4e1c-bce7-5cce8285e228" containerName="maas-api" Jul 10 03:14:17.261962 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.261871 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8242ca35-77eb-4e1c-bce7-5cce8285e228" containerName="maas-api" Jul 10 03:14:17.266207 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.266190 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.269322 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.269301 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-5374048d-serving-cert\"" Jul 10 03:14:17.279782 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.279751 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv"] Jul 10 03:14:17.280673 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.280620 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:17.372389 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.372352 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:17.417693 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.417631 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2z96\" (UniqueName: \"kubernetes.io/projected/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-kube-api-access-t2z96\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.417885 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.417844 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-openshift-service-ca\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.417943 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.417887 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-maas-api-tls\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.473214 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.473179 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:17.518967 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.518928 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-openshift-service-ca\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.518967 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.518970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-maas-api-tls\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.519178 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.519010 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2z96\" (UniqueName: \"kubernetes.io/projected/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-kube-api-access-t2z96\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.519733 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.519706 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-openshift-service-ca\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.521464 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.521442 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-maas-api-tls\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.530493 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.530468 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2z96\" (UniqueName: \"kubernetes.io/projected/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-kube-api-access-t2z96\") pod \"maas-api-e2e-derive-5374048d-5bc7548996-56lfv\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.578124 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.578091 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:17.726736 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.726712 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv"] Jul 10 03:14:17.728540 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:14:17.728514 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod104ddbf2_8c2a_42ee_b72f_c344fb1a0507.slice/crio-a663b31b42731560e4abca2d811cef868b158acd2263165714c6e5493aa31cab WatchSource:0}: Error finding container a663b31b42731560e4abca2d811cef868b158acd2263165714c6e5493aa31cab: Status 404 returned error can't find the container with id a663b31b42731560e4abca2d811cef868b158acd2263165714c6e5493aa31cab Jul 10 03:14:17.897667 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:17.897616 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:18.000340 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:18.000242 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:18.236633 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:18.236600 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" event={"ID":"104ddbf2-8c2a-42ee-b72f-c344fb1a0507","Type":"ContainerStarted","Data":"a663b31b42731560e4abca2d811cef868b158acd2263165714c6e5493aa31cab"} Jul 10 03:14:19.244311 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:19.244257 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" event={"ID":"104ddbf2-8c2a-42ee-b72f-c344fb1a0507","Type":"ContainerStarted","Data":"c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652"} Jul 10 03:14:19.244794 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:19.244424 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:19.262398 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:19.262349 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" podStartSLOduration=1.7334272240000002 podStartE2EDuration="2.262333561s" totalImagesPullingTime="528.906337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:14:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:14:19.261075095 +0000 UTC m=+2065.969883631" watchObservedRunningTime="2026-07-10 03:14:19.262333561 +0000 UTC m=+2065.971142096" Jul 10 03:14:21.777940 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:21.777905 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv"] Jul 10 03:14:21.778381 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:21.778122 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" podUID="104ddbf2-8c2a-42ee-b72f-c344fb1a0507" containerName="maas-api" containerID="cri-o://c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652" gracePeriod=30 Jul 10 03:14:21.813235 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:21.813206 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:22.035123 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.035072 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:22.060428 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.060401 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2z96\" (UniqueName: \"kubernetes.io/projected/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-kube-api-access-t2z96\") pod \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " Jul 10 03:14:22.060580 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.060449 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-openshift-service-ca\") pod \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " Jul 10 03:14:22.060580 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.060509 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-maas-api-tls\") pod \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\" (UID: \"104ddbf2-8c2a-42ee-b72f-c344fb1a0507\") " Jul 10 03:14:22.061165 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.061142 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "104ddbf2-8c2a-42ee-b72f-c344fb1a0507" (UID: "104ddbf2-8c2a-42ee-b72f-c344fb1a0507"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:14:22.062998 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.062955 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-kube-api-access-t2z96" (OuterVolumeSpecName: "kube-api-access-t2z96") pod "104ddbf2-8c2a-42ee-b72f-c344fb1a0507" (UID: "104ddbf2-8c2a-42ee-b72f-c344fb1a0507"). InnerVolumeSpecName "kube-api-access-t2z96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:14:22.063610 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.063583 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "104ddbf2-8c2a-42ee-b72f-c344fb1a0507" (UID: "104ddbf2-8c2a-42ee-b72f-c344fb1a0507"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:14:22.066377 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.066356 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:14:22.161978 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.161950 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:14:22.161978 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.161975 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2z96\" (UniqueName: \"kubernetes.io/projected/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-kube-api-access-t2z96\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:14:22.161978 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.161985 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/104ddbf2-8c2a-42ee-b72f-c344fb1a0507-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:14:22.261205 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.261170 2585 generic.go:356] "Generic (PLEG): container finished" podID="104ddbf2-8c2a-42ee-b72f-c344fb1a0507" containerID="c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652" exitCode=0 Jul 10 03:14:22.261381 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.261228 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" Jul 10 03:14:22.261381 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.261239 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" event={"ID":"104ddbf2-8c2a-42ee-b72f-c344fb1a0507","Type":"ContainerDied","Data":"c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652"} Jul 10 03:14:22.261381 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.261279 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv" event={"ID":"104ddbf2-8c2a-42ee-b72f-c344fb1a0507","Type":"ContainerDied","Data":"a663b31b42731560e4abca2d811cef868b158acd2263165714c6e5493aa31cab"} Jul 10 03:14:22.261381 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.261300 2585 scope.go:117] "RemoveContainer" containerID="c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652" Jul 10 03:14:22.270024 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.270004 2585 scope.go:117] "RemoveContainer" containerID="c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652" Jul 10 03:14:22.270277 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:14:22.270260 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652\": container with ID starting with c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652 not found: ID does not exist" containerID="c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652" Jul 10 03:14:22.270327 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.270284 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652"} err="failed to get container status \"c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652\": rpc error: code = NotFound desc = could not find container \"c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652\": container with ID starting with c70fe7623a9cd18d4a85ac6d00a3932443047dc15ab705e5dd804d8eba9ae652 not found: ID does not exist" Jul 10 03:14:22.282706 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.282680 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv"] Jul 10 03:14:22.295584 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.295525 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5374048d-5bc7548996-56lfv"] Jul 10 03:14:22.889495 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:22.889462 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:23.668112 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:23.668074 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:24.112480 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:24.112452 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="104ddbf2-8c2a-42ee-b72f-c344fb1a0507" path="/var/lib/kubelet/pods/104ddbf2-8c2a-42ee-b72f-c344fb1a0507/volumes" Jul 10 03:14:28.158478 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:28.158435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:28.228915 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:28.228886 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:28.678911 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:28.678878 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:28.776127 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:28.776095 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:28.878202 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:28.878158 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:29.459721 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:29.459688 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:14:30.450013 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:30.449981 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:36.320544 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:36.320512 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:36.428093 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:36.428058 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:36.532674 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:36.532615 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:36.630553 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:36.630464 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:37.095785 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:37.095733 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:14:53.979623 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:53.979596 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:14:53.986294 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:14:53.986270 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:15:03.184560 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:03.184524 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:08.994950 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:08.994917 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:09.117691 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:09.117662 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:09.232908 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:09.232870 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:25.243527 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:25.243494 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:25.762796 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:25.762762 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:25.860459 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:25.860419 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:31.169393 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:31.169322 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:31.289216 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:31.289187 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:31.382472 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:31.382439 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:41.968211 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:41.968121 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:42.827686 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:42.827611 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:48.738525 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:48.738415 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:48.820961 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:48.820930 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:48.909829 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:48.909799 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:49.008643 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:49.008566 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:49.110856 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:49.110808 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:49.922374 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:49.922337 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:50.023354 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:50.023276 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:50.131670 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:50.131620 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:51.052941 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:51.052902 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:15:51.895884 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:51.895819 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:57.822702 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:57.822663 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:57.895998 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:57.895966 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:15:58.562888 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:15:58.562858 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:00.288853 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:00.288815 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:16:04.436185 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:04.436149 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:04.505586 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:04.505552 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:04.712354 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:04.712239 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:05.050675 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:05.050547 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:05.580503 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:05.580475 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:08.570189 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.570155 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:08.662025 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.661990 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8"] Jul 10 03:16:08.662608 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.662592 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="104ddbf2-8c2a-42ee-b72f-c344fb1a0507" containerName="maas-api" Jul 10 03:16:08.662685 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.662612 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="104ddbf2-8c2a-42ee-b72f-c344fb1a0507" containerName="maas-api" Jul 10 03:16:08.662769 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.662757 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="104ddbf2-8c2a-42ee-b72f-c344fb1a0507" containerName="maas-api" Jul 10 03:16:08.666056 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.666029 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.668274 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.668234 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:08.668787 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.668770 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1924c46c-serving-cert\"" Jul 10 03:16:08.679638 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.679607 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8"] Jul 10 03:16:08.740292 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.740246 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-maas-api-tls\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.740292 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.740295 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmrf\" (UniqueName: \"kubernetes.io/projected/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-kube-api-access-htmrf\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.740526 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.740401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-openshift-service-ca\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.766801 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.766761 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:08.841399 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.841298 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-maas-api-tls\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.841399 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.841343 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmrf\" (UniqueName: \"kubernetes.io/projected/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-kube-api-access-htmrf\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.841399 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.841388 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-openshift-service-ca\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.842203 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.842182 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-openshift-service-ca\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.843928 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.843910 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-maas-api-tls\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.851553 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.851519 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmrf\" (UniqueName: \"kubernetes.io/projected/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-kube-api-access-htmrf\") pod \"maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:08.866912 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.866872 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:08.977600 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:08.977556 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:09.132671 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.130598 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8"] Jul 10 03:16:09.132671 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.131250 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 03:16:09.267139 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.267102 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:09.367457 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.367425 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:09.863360 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.863266 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" event={"ID":"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d","Type":"ContainerStarted","Data":"08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134"} Jul 10 03:16:09.863360 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.863330 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" event={"ID":"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d","Type":"ContainerStarted","Data":"e28c040a3b2e8aaa61ea26366c53e580c0b1b6b343da3ddd3f05746edfdde42e"} Jul 10 03:16:09.863770 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.863415 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:09.885931 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:09.885878 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" podStartSLOduration=1.466609248 podStartE2EDuration="1.885859936s" totalImagesPullingTime="419.250688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:16:09.884089487 +0000 UTC m=+2176.592898037" watchObservedRunningTime="2026-07-10 03:16:09.885859936 +0000 UTC m=+2176.594668472" Jul 10 03:16:13.104754 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:13.104719 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:13.877331 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:13.877295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:14.045768 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:14.045729 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:14.311101 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:14.311068 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:14.460617 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:14.460580 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:16:15.874536 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:15.874503 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:16:20.663210 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:20.663166 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:16:20.758791 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:16:20.758756 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:21.461985 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:21.461946 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:26.550272 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.550226 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8"] Jul 10 03:17:26.550773 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.550455 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" podUID="fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" containerName="maas-api" containerID="cri-o://08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134" gracePeriod=30 Jul 10 03:17:26.573196 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.573157 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:26.811101 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.811031 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:17:26.874889 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.874848 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-openshift-service-ca\") pod \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " Jul 10 03:17:26.875067 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.874910 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-maas-api-tls\") pod \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " Jul 10 03:17:26.875067 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.874950 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htmrf\" (UniqueName: \"kubernetes.io/projected/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-kube-api-access-htmrf\") pod \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\" (UID: \"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d\") " Jul 10 03:17:26.875272 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.875234 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" (UID: "fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:17:26.877067 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.877037 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" (UID: "fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:17:26.877168 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.877070 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-kube-api-access-htmrf" (OuterVolumeSpecName: "kube-api-access-htmrf") pod "fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" (UID: "fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d"). InnerVolumeSpecName "kube-api-access-htmrf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:17:26.976413 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.976369 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:17:26.976413 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.976403 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:17:26.976413 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:26.976416 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htmrf\" (UniqueName: \"kubernetes.io/projected/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d-kube-api-access-htmrf\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:17:27.180049 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.180019 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:27.300235 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.300196 2585 generic.go:356] "Generic (PLEG): container finished" podID="fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" containerID="08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134" exitCode=0 Jul 10 03:17:27.300408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.300262 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" Jul 10 03:17:27.300408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.300280 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" event={"ID":"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d","Type":"ContainerDied","Data":"08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134"} Jul 10 03:17:27.300408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.300335 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8" event={"ID":"fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d","Type":"ContainerDied","Data":"e28c040a3b2e8aaa61ea26366c53e580c0b1b6b343da3ddd3f05746edfdde42e"} Jul 10 03:17:27.300408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.300366 2585 scope.go:117] "RemoveContainer" containerID="08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134" Jul 10 03:17:27.310427 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.310409 2585 scope.go:117] "RemoveContainer" containerID="08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134" Jul 10 03:17:27.310700 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:17:27.310679 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134\": container with ID starting with 08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134 not found: ID does not exist" containerID="08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134" Jul 10 03:17:27.310755 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.310709 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134"} err="failed to get container status \"08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134\": rpc error: code = NotFound desc = could not find container \"08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134\": container with ID starting with 08f1087c1fedc0a5fc5444e2fd1e568ee0406fb181d3df1d6f92dd0dd12f0134 not found: ID does not exist" Jul 10 03:17:27.323430 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.323395 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8"] Jul 10 03:17:27.326757 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:27.326734 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1924c46c-69b789fc59-dvtk8"] Jul 10 03:17:28.112630 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:28.112596 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" path="/var/lib/kubelet/pods/fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d/volumes" Jul 10 03:17:30.049477 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:30.049440 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:17:32.975310 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:32.975277 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:33.032796 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:33.032764 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:33.273570 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:33.273478 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:33.521186 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:33.521112 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:33.653468 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:33.653368 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:33.752331 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:33.752297 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:33.911875 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:33.911767 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:39.688032 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:39.687996 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:39.747591 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:39.747559 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:39.825192 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:39.825157 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:39.924787 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:39.924753 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:40.067147 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:40.067060 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:46.160369 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:46.160329 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:46.235908 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:46.235878 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:53.143687 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:53.143629 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:53.473578 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:53.473533 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:53.537844 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:53.537799 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:56.084156 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.084122 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:56.183391 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.183355 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:56.265554 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.265517 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x"] Jul 10 03:17:56.266159 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.266138 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" containerName="maas-api" Jul 10 03:17:56.266159 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.266159 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" containerName="maas-api" Jul 10 03:17:56.266326 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.266268 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdb62fb2-1af8-4c73-a04a-8cf7b2a6797d" containerName="maas-api" Jul 10 03:17:56.271144 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.271121 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.273883 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.273862 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-57198657-serving-cert\"" Jul 10 03:17:56.279042 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.279016 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x"] Jul 10 03:17:56.298985 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.298944 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:56.355378 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.355282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnnz6\" (UniqueName: \"kubernetes.io/projected/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-kube-api-access-bnnz6\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.355378 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.355363 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.355579 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.355400 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-openshift-service-ca\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.456259 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.456220 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.456461 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.456273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-openshift-service-ca\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.456461 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.456313 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnnz6\" (UniqueName: \"kubernetes.io/projected/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-kube-api-access-bnnz6\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.456461 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:17:56.456352 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-57198657-serving-cert: secret "maas-api-e2e-mt-57198657-serving-cert" not found Jul 10 03:17:56.456461 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:17:56.456415 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls podName:daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e nodeName:}" failed. No retries permitted until 2026-07-10 03:17:56.956398288 +0000 UTC m=+2283.665206802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls") pod "maas-api-e2e-mt-57198657-679d74fbbc-dr52x" (UID: "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e") : secret "maas-api-e2e-mt-57198657-serving-cert" not found Jul 10 03:17:56.457048 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.457029 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-openshift-service-ca\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.467279 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.467252 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnnz6\" (UniqueName: \"kubernetes.io/projected/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-kube-api-access-bnnz6\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.764237 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.764205 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:56.863965 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.863934 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:56.961827 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.961790 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.964311 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.964290 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls\") pod \"maas-api-e2e-mt-57198657-679d74fbbc-dr52x\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:56.967967 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:56.967941 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:17:57.183497 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:57.183461 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:57.336515 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:57.336487 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x"] Jul 10 03:17:57.338109 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:17:57.338079 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaca4cc4_7f4b_4e6b_a3e9_a500b1adee4e.slice/crio-3a5f641cf3a91a2e16a263053c381b1146c5b177c5315c608b37ef2bc33c496b WatchSource:0}: Error finding container 3a5f641cf3a91a2e16a263053c381b1146c5b177c5315c608b37ef2bc33c496b: Status 404 returned error can't find the container with id 3a5f641cf3a91a2e16a263053c381b1146c5b177c5315c608b37ef2bc33c496b Jul 10 03:17:57.474433 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:57.474327 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" event={"ID":"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e","Type":"ContainerStarted","Data":"3a5f641cf3a91a2e16a263053c381b1146c5b177c5315c608b37ef2bc33c496b"} Jul 10 03:17:58.481128 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:58.481091 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" event={"ID":"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e","Type":"ContainerStarted","Data":"e85a5e3866e18c8d175958bf2ab3edabc0280d9fee376fa23f784145efb1cac9"} Jul 10 03:17:58.481624 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:58.481155 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:17:58.500581 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:17:58.500526 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" podStartSLOduration=2.122767012 podStartE2EDuration="2.500509499s" totalImagesPullingTime="377.742487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:17:58.4992917 +0000 UTC m=+2285.208100237" watchObservedRunningTime="2026-07-10 03:17:58.500509499 +0000 UTC m=+2285.209318034" Jul 10 03:18:00.618910 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:18:00.618862 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:18:03.450594 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:18:03.450554 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:18:04.492583 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:18:04.492538 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:18:10.485682 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:18:10.485630 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:18:15.757241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:18:15.757201 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:18:46.164848 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:18:46.164750 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:19:17.924000 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:19:17.923963 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:19:18.024899 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:19:18.024868 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:19:54.023756 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:19:54.023727 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:19:54.033022 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:19:54.032998 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:20:26.521798 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:20:26.521755 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:20:27.718156 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:20:27.718124 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:20:34.719073 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:20:34.719034 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:20:34.767964 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:20:34.767925 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:21:17.623467 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:21:17.623434 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:23:18.866611 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:23:18.866533 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:23:18.955698 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:23:18.955638 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:23:57.009017 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:23:57.008962 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:23:58.945205 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:23:58.945162 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:24:02.089809 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:02.089777 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:38.134108 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.134073 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:38.621578 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.621521 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x"] Jul 10 03:24:38.621906 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.621879 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" podUID="daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" containerName="maas-api" containerID="cri-o://e85a5e3866e18c8d175958bf2ab3edabc0280d9fee376fa23f784145efb1cac9" gracePeriod=30 Jul 10 03:24:38.637853 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.637815 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:38.749955 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.749928 2585 generic.go:356] "Generic (PLEG): container finished" podID="daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" containerID="e85a5e3866e18c8d175958bf2ab3edabc0280d9fee376fa23f784145efb1cac9" exitCode=0 Jul 10 03:24:38.750096 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.750001 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" event={"ID":"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e","Type":"ContainerDied","Data":"e85a5e3866e18c8d175958bf2ab3edabc0280d9fee376fa23f784145efb1cac9"} Jul 10 03:24:38.874475 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.874405 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:24:38.953289 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.953252 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnnz6\" (UniqueName: \"kubernetes.io/projected/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-kube-api-access-bnnz6\") pod \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " Jul 10 03:24:38.953493 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.953311 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-openshift-service-ca\") pod \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " Jul 10 03:24:38.953493 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.953361 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls\") pod \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\" (UID: \"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e\") " Jul 10 03:24:38.953761 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.953734 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" (UID: "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:24:38.955443 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.955416 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" (UID: "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:24:38.955499 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:38.955432 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-kube-api-access-bnnz6" (OuterVolumeSpecName: "kube-api-access-bnnz6") pod "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" (UID: "daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e"). InnerVolumeSpecName "kube-api-access-bnnz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:24:39.054720 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.054679 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:24:39.054720 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.054712 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnnz6\" (UniqueName: \"kubernetes.io/projected/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-kube-api-access-bnnz6\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:24:39.054720 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.054721 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:24:39.383271 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.383238 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:39.757187 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.757150 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" event={"ID":"daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e","Type":"ContainerDied","Data":"3a5f641cf3a91a2e16a263053c381b1146c5b177c5315c608b37ef2bc33c496b"} Jul 10 03:24:39.757187 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.757173 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x" Jul 10 03:24:39.757503 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.757219 2585 scope.go:117] "RemoveContainer" containerID="e85a5e3866e18c8d175958bf2ab3edabc0280d9fee376fa23f784145efb1cac9" Jul 10 03:24:39.779404 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.779372 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x"] Jul 10 03:24:39.785792 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:39.785763 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57198657-679d74fbbc-dr52x"] Jul 10 03:24:40.112388 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:40.112308 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" path="/var/lib/kubelet/pods/daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e/volumes" Jul 10 03:24:42.247352 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:42.247309 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:24:45.194019 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:45.193984 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:45.249489 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:45.249401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:45.498269 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:45.498181 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:45.757445 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:45.757353 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:49.851191 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:49.851159 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:49.950088 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:49.950054 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:50.050411 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:50.050372 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:50.836063 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:50.836018 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:50.932326 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:50.932297 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:51.032280 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:51.032239 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:52.538261 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:52.538222 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:54.060768 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:54.060738 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:24:54.070509 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:54.070483 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:24:54.249049 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:54.249006 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:24:58.184454 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:58.184415 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:58.238796 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:58.238766 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:24:58.877459 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:24:58.877424 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:04.778670 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:04.778614 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:04.882894 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:04.882864 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:04.982698 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:04.982665 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:05.254185 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:05.254143 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:25:15.541762 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:15.541725 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:15.923538 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:15.923449 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:27.408693 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:27.408609 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:27.506176 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:27.506137 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:35.257473 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:35.257435 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:36.365604 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:36.365572 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:36.920184 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:36.920149 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:41.111921 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:41.111887 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:42.591919 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:42.591882 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:42.694641 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:42.694605 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:42.969910 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:42.969854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:43.309769 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:43.309685 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:43.441196 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:43.441168 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:46.553104 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.553071 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:46.648334 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.648299 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:46.654745 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.654715 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt"] Jul 10 03:25:46.655136 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.655124 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" containerName="maas-api" Jul 10 03:25:46.655185 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.655138 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" containerName="maas-api" Jul 10 03:25:46.655221 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.655212 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="daca4cc4-7f4b-4e6b-a3e9-a500b1adee4e" containerName="maas-api" Jul 10 03:25:46.657442 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.657422 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.660894 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.660860 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e7f792-serving-cert\"" Jul 10 03:25:46.674929 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.674899 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt"] Jul 10 03:25:46.732913 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.732878 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:46.776169 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.776137 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.776361 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.776190 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76zx9\" (UniqueName: \"kubernetes.io/projected/d89590e5-1f63-4db9-819b-541c11a39d5d-kube-api-access-76zx9\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.776361 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.776314 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89590e5-1f63-4db9-819b-541c11a39d5d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.877523 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.877413 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89590e5-1f63-4db9-819b-541c11a39d5d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.877523 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.877504 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.877887 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.877678 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76zx9\" (UniqueName: \"kubernetes.io/projected/d89590e5-1f63-4db9-819b-541c11a39d5d-kube-api-access-76zx9\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.877887 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:25:46.877770 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-serving-cert: secret "maas-api-e2e-inf-a-e7f792-serving-cert" not found Jul 10 03:25:46.877887 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:25:46.877844 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls podName:d89590e5-1f63-4db9-819b-541c11a39d5d nodeName:}" failed. No retries permitted until 2026-07-10 03:25:47.377822123 +0000 UTC m=+2754.086630651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls") pod "maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" (UID: "d89590e5-1f63-4db9-819b-541c11a39d5d") : secret "maas-api-e2e-inf-a-e7f792-serving-cert" not found Jul 10 03:25:46.878385 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.878358 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89590e5-1f63-4db9-819b-541c11a39d5d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.887519 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.887489 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76zx9\" (UniqueName: \"kubernetes.io/projected/d89590e5-1f63-4db9-819b-541c11a39d5d-kube-api-access-76zx9\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:46.930224 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:46.930184 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:47.260264 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.260231 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:47.372376 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.372346 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:47.382582 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.382549 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:47.385034 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.385017 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls\") pod \"maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:47.568149 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.568046 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:47.720614 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.720580 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt"] Jul 10 03:25:47.722999 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:25:47.722966 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89590e5_1f63_4db9_819b_541c11a39d5d.slice/crio-a26492c481c0b9bd1cae033427903c18841ea59765cfc17cb57e2b6fcbf3e36a WatchSource:0}: Error finding container a26492c481c0b9bd1cae033427903c18841ea59765cfc17cb57e2b6fcbf3e36a: Status 404 returned error can't find the container with id a26492c481c0b9bd1cae033427903c18841ea59765cfc17cb57e2b6fcbf3e36a Jul 10 03:25:47.724284 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:47.724263 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 03:25:48.146182 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:48.146147 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" event={"ID":"d89590e5-1f63-4db9-819b-541c11a39d5d","Type":"ContainerStarted","Data":"a26492c481c0b9bd1cae033427903c18841ea59765cfc17cb57e2b6fcbf3e36a"} Jul 10 03:25:49.153736 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:49.153698 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" event={"ID":"d89590e5-1f63-4db9-819b-541c11a39d5d","Type":"ContainerStarted","Data":"ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053"} Jul 10 03:25:49.154149 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:49.153864 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:49.171641 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:49.171582 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" podStartSLOduration=2.618249618 podStartE2EDuration="3.171562453s" totalImagesPullingTime="553.312835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:25:49.169497635 +0000 UTC m=+2755.878306172" watchObservedRunningTime="2026-07-10 03:25:49.171562453 +0000 UTC m=+2755.880370989" Jul 10 03:25:51.074081 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:51.074041 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:51.437931 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:51.437904 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:51.784813 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:51.784721 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:51.930562 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:51.930524 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:54.546133 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.546097 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:54.647441 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.647401 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:54.764782 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.764743 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:25:54.769220 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.769188 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr"] Jul 10 03:25:54.773812 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.773774 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.776733 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.776695 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-48f01f-serving-cert\"" Jul 10 03:25:54.783336 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.783306 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr"] Jul 10 03:25:54.806295 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.806211 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:54.856334 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.856291 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7134417-ea8d-4722-a71c-f7162e3ea3bc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.856520 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.856406 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srxgv\" (UniqueName: \"kubernetes.io/projected/c7134417-ea8d-4722-a71c-f7162e3ea3bc-kube-api-access-srxgv\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.856520 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.856436 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7134417-ea8d-4722-a71c-f7162e3ea3bc-maas-api-tls\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.911169 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.911138 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:54.957518 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.957477 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7134417-ea8d-4722-a71c-f7162e3ea3bc-maas-api-tls\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.957727 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.957550 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7134417-ea8d-4722-a71c-f7162e3ea3bc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.957727 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.957625 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srxgv\" (UniqueName: \"kubernetes.io/projected/c7134417-ea8d-4722-a71c-f7162e3ea3bc-kube-api-access-srxgv\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.958318 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.958294 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7134417-ea8d-4722-a71c-f7162e3ea3bc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.960068 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.960045 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7134417-ea8d-4722-a71c-f7162e3ea3bc-maas-api-tls\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:54.970081 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:54.970048 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srxgv\" (UniqueName: \"kubernetes.io/projected/c7134417-ea8d-4722-a71c-f7162e3ea3bc-kube-api-access-srxgv\") pod \"maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:55.088939 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:55.088835 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:55.167115 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:55.167085 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:25:55.246617 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:55.246580 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr"] Jul 10 03:25:55.248583 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:25:55.248546 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7134417_ea8d_4722_a71c_f7162e3ea3bc.slice/crio-05e88c587596a5e0719e0b1432054bea9de64239daeb0d9a91a527ab3c07d8f9 WatchSource:0}: Error finding container 05e88c587596a5e0719e0b1432054bea9de64239daeb0d9a91a527ab3c07d8f9: Status 404 returned error can't find the container with id 05e88c587596a5e0719e0b1432054bea9de64239daeb0d9a91a527ab3c07d8f9 Jul 10 03:25:55.271595 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:55.271568 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:55.371037 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:55.370953 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:25:56.202679 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:56.202621 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" event={"ID":"c7134417-ea8d-4722-a71c-f7162e3ea3bc","Type":"ContainerStarted","Data":"a87f843b033f24a7b757683c7dae7c00d37cf1fd3c62f794bd9b449a671dec28"} Jul 10 03:25:56.203169 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:56.202710 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" event={"ID":"c7134417-ea8d-4722-a71c-f7162e3ea3bc","Type":"ContainerStarted","Data":"05e88c587596a5e0719e0b1432054bea9de64239daeb0d9a91a527ab3c07d8f9"} Jul 10 03:25:56.203169 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:56.202833 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:25:56.221867 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:56.221814 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" podStartSLOduration=1.84213277 podStartE2EDuration="2.221795916s" totalImagesPullingTime="379.663146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:25:56.219920645 +0000 UTC m=+2762.928729181" watchObservedRunningTime="2026-07-10 03:25:56.221795916 +0000 UTC m=+2762.930604458" Jul 10 03:25:59.129375 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:25:59.129330 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:02.214432 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:02.214402 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:26:06.777490 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:06.777451 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:26:08.045135 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:08.045101 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:08.145733 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:08.145665 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:17.014626 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:17.014538 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:17.115460 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:17.115432 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:18.073003 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:18.072973 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:20.754661 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:20.754621 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:26:32.948495 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:32.948454 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:26:39.781500 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:39.781454 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:26:45.154919 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:26:45.154879 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:27:12.716386 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:12.716331 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:13.045021 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:13.044897 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:13.575751 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:13.575712 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:15.983391 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:15.983356 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:16.081440 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.081408 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:16.170059 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.170027 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9"] Jul 10 03:27:16.173669 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.173631 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.176578 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.176555 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-0566e7-serving-cert\"" Jul 10 03:27:16.186405 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.186377 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9"] Jul 10 03:27:16.199711 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.199672 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp5nk\" (UniqueName: \"kubernetes.io/projected/bb754d3b-b48e-4e5b-af4e-04439fe1d391-kube-api-access-fp5nk\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.199935 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.199759 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.200017 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.199930 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb754d3b-b48e-4e5b-af4e-04439fe1d391-openshift-service-ca\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.211685 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.211621 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:16.300750 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.300637 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb754d3b-b48e-4e5b-af4e-04439fe1d391-openshift-service-ca\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.300750 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.300710 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp5nk\" (UniqueName: \"kubernetes.io/projected/bb754d3b-b48e-4e5b-af4e-04439fe1d391-kube-api-access-fp5nk\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.300932 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.300750 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.300932 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:27:16.300897 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-serving-cert: secret "maas-api-e2e-reuse-0566e7-serving-cert" not found Jul 10 03:27:16.301003 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:27:16.300965 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls podName:bb754d3b-b48e-4e5b-af4e-04439fe1d391 nodeName:}" failed. No retries permitted until 2026-07-10 03:27:16.800947581 +0000 UTC m=+2843.509756095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls") pod "maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" (UID: "bb754d3b-b48e-4e5b-af4e-04439fe1d391") : secret "maas-api-e2e-reuse-0566e7-serving-cert" not found Jul 10 03:27:16.301481 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.301458 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb754d3b-b48e-4e5b-af4e-04439fe1d391-openshift-service-ca\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.309632 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.309604 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp5nk\" (UniqueName: \"kubernetes.io/projected/bb754d3b-b48e-4e5b-af4e-04439fe1d391-kube-api-access-fp5nk\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.691137 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.691099 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:16.793627 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.793591 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:16.805946 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.805915 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.808760 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.808721 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls\") pod \"maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:16.891891 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:16.891854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:17.085532 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:17.085433 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:17.238058 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:17.238027 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9"] Jul 10 03:27:17.240425 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:27:17.240393 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb754d3b_b48e_4e5b_af4e_04439fe1d391.slice/crio-37eec5630aad502221b5652c721068128e15bd2af987ad3749bce7ed3f1cacbe WatchSource:0}: Error finding container 37eec5630aad502221b5652c721068128e15bd2af987ad3749bce7ed3f1cacbe: Status 404 returned error can't find the container with id 37eec5630aad502221b5652c721068128e15bd2af987ad3749bce7ed3f1cacbe Jul 10 03:27:17.670926 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:17.670889 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" event={"ID":"bb754d3b-b48e-4e5b-af4e-04439fe1d391","Type":"ContainerStarted","Data":"37eec5630aad502221b5652c721068128e15bd2af987ad3749bce7ed3f1cacbe"} Jul 10 03:27:18.678267 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:18.678224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" event={"ID":"bb754d3b-b48e-4e5b-af4e-04439fe1d391","Type":"ContainerStarted","Data":"a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940"} Jul 10 03:27:18.678695 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:18.678298 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:18.696783 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:18.696719 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" podStartSLOduration=2.31809904 podStartE2EDuration="2.696697534s" totalImagesPullingTime="378.598494ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:27:18.693942903 +0000 UTC m=+2845.402751439" watchObservedRunningTime="2026-07-10 03:27:18.696697534 +0000 UTC m=+2845.405506071" Jul 10 03:27:20.522862 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:20.522824 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:22.044953 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:22.044903 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:27:24.690132 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:24.690101 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:27:27.254575 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:27.254540 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:27:46.547434 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:27:46.547340 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:28:39.831515 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:28:39.831475 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:28:40.992016 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:28:40.991983 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:28:41.471922 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:28:41.471889 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:28:46.555453 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:28:46.555417 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:19.630334 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:19.630253 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:22.726015 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:22.725978 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9"] Jul 10 03:29:22.726571 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:22.726265 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" podUID="bb754d3b-b48e-4e5b-af4e-04439fe1d391" containerName="maas-api" containerID="cri-o://a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940" gracePeriod=30 Jul 10 03:29:22.746401 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:22.746375 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:22.982858 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:22.982783 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:29:23.086763 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.086728 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp5nk\" (UniqueName: \"kubernetes.io/projected/bb754d3b-b48e-4e5b-af4e-04439fe1d391-kube-api-access-fp5nk\") pod \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " Jul 10 03:29:23.086924 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.086815 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb754d3b-b48e-4e5b-af4e-04439fe1d391-openshift-service-ca\") pod \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " Jul 10 03:29:23.086924 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.086837 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls\") pod \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\" (UID: \"bb754d3b-b48e-4e5b-af4e-04439fe1d391\") " Jul 10 03:29:23.087170 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.087138 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb754d3b-b48e-4e5b-af4e-04439fe1d391-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb754d3b-b48e-4e5b-af4e-04439fe1d391" (UID: "bb754d3b-b48e-4e5b-af4e-04439fe1d391"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:29:23.088817 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.088792 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb754d3b-b48e-4e5b-af4e-04439fe1d391-kube-api-access-fp5nk" (OuterVolumeSpecName: "kube-api-access-fp5nk") pod "bb754d3b-b48e-4e5b-af4e-04439fe1d391" (UID: "bb754d3b-b48e-4e5b-af4e-04439fe1d391"). InnerVolumeSpecName "kube-api-access-fp5nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:29:23.088908 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.088829 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb754d3b-b48e-4e5b-af4e-04439fe1d391" (UID: "bb754d3b-b48e-4e5b-af4e-04439fe1d391"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:29:23.187856 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.187814 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp5nk\" (UniqueName: \"kubernetes.io/projected/bb754d3b-b48e-4e5b-af4e-04439fe1d391-kube-api-access-fp5nk\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:29:23.187856 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.187844 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb754d3b-b48e-4e5b-af4e-04439fe1d391-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:29:23.187856 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.187862 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb754d3b-b48e-4e5b-af4e-04439fe1d391-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:29:23.373522 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.373431 2585 generic.go:356] "Generic (PLEG): container finished" podID="bb754d3b-b48e-4e5b-af4e-04439fe1d391" containerID="a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940" exitCode=0 Jul 10 03:29:23.373522 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.373476 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" Jul 10 03:29:23.373749 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.373520 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" event={"ID":"bb754d3b-b48e-4e5b-af4e-04439fe1d391","Type":"ContainerDied","Data":"a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940"} Jul 10 03:29:23.373749 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.373567 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9" event={"ID":"bb754d3b-b48e-4e5b-af4e-04439fe1d391","Type":"ContainerDied","Data":"37eec5630aad502221b5652c721068128e15bd2af987ad3749bce7ed3f1cacbe"} Jul 10 03:29:23.373749 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.373589 2585 scope.go:117] "RemoveContainer" containerID="a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940" Jul 10 03:29:23.383783 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.383762 2585 scope.go:117] "RemoveContainer" containerID="a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940" Jul 10 03:29:23.384120 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:29:23.384098 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940\": container with ID starting with a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940 not found: ID does not exist" containerID="a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940" Jul 10 03:29:23.384208 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.384126 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940"} err="failed to get container status \"a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940\": rpc error: code = NotFound desc = could not find container \"a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940\": container with ID starting with a02a56fb7b53c6b04cd50448bde57961f7f5746064875ffa6708ef76b9f25940 not found: ID does not exist" Jul 10 03:29:23.405772 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.405707 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9"] Jul 10 03:29:23.407909 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.407876 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-0566e7-7cbfd7fc7-wbbg9"] Jul 10 03:29:23.430005 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.429950 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:23.602977 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:23.602939 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:24.112551 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:24.112521 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb754d3b-b48e-4e5b-af4e-04439fe1d391" path="/var/lib/kubelet/pods/bb754d3b-b48e-4e5b-af4e-04439fe1d391/volumes" Jul 10 03:29:28.650802 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:28.650764 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:29:29.129867 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:29.129835 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:29.192330 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:29.192299 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:29.438533 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:29.438504 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:29.688311 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:29.688274 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:33.546525 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:33.546493 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:37.455998 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:37.455965 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:38.194801 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:38.194761 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:38.295501 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:38.295462 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:29:48.944006 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:48.943968 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:29:54.096774 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:54.096737 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:29:54.109678 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:29:54.109633 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:30:28.161808 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:28.161777 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:28.262348 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:28.262313 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:37.946826 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:37.946791 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:43.300179 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.300140 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt"] Jul 10 03:30:43.300691 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.300414 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" podUID="d89590e5-1f63-4db9-819b-541c11a39d5d" containerName="maas-api" containerID="cri-o://ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053" gracePeriod=30 Jul 10 03:30:43.332476 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.332450 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:43.552779 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.552717 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:30:43.634950 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.634914 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89590e5-1f63-4db9-819b-541c11a39d5d-openshift-service-ca\") pod \"d89590e5-1f63-4db9-819b-541c11a39d5d\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " Jul 10 03:30:43.635142 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.634987 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls\") pod \"d89590e5-1f63-4db9-819b-541c11a39d5d\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " Jul 10 03:30:43.635142 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.635024 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76zx9\" (UniqueName: \"kubernetes.io/projected/d89590e5-1f63-4db9-819b-541c11a39d5d-kube-api-access-76zx9\") pod \"d89590e5-1f63-4db9-819b-541c11a39d5d\" (UID: \"d89590e5-1f63-4db9-819b-541c11a39d5d\") " Jul 10 03:30:43.635340 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.635307 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89590e5-1f63-4db9-819b-541c11a39d5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d89590e5-1f63-4db9-819b-541c11a39d5d" (UID: "d89590e5-1f63-4db9-819b-541c11a39d5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:30:43.637117 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.637078 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d89590e5-1f63-4db9-819b-541c11a39d5d" (UID: "d89590e5-1f63-4db9-819b-541c11a39d5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:30:43.637339 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.637313 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89590e5-1f63-4db9-819b-541c11a39d5d-kube-api-access-76zx9" (OuterVolumeSpecName: "kube-api-access-76zx9") pod "d89590e5-1f63-4db9-819b-541c11a39d5d" (UID: "d89590e5-1f63-4db9-819b-541c11a39d5d"). InnerVolumeSpecName "kube-api-access-76zx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:30:43.736461 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.736424 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d89590e5-1f63-4db9-819b-541c11a39d5d-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:30:43.736461 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.736456 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76zx9\" (UniqueName: \"kubernetes.io/projected/d89590e5-1f63-4db9-819b-541c11a39d5d-kube-api-access-76zx9\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:30:43.736461 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.736466 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d89590e5-1f63-4db9-819b-541c11a39d5d-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:30:43.737988 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.737960 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:43.838408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.838317 2585 generic.go:356] "Generic (PLEG): container finished" podID="d89590e5-1f63-4db9-819b-541c11a39d5d" containerID="ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053" exitCode=0 Jul 10 03:30:43.838408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.838364 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" Jul 10 03:30:43.838408 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.838389 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" event={"ID":"d89590e5-1f63-4db9-819b-541c11a39d5d","Type":"ContainerDied","Data":"ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053"} Jul 10 03:30:43.838699 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.838442 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt" event={"ID":"d89590e5-1f63-4db9-819b-541c11a39d5d","Type":"ContainerDied","Data":"a26492c481c0b9bd1cae033427903c18841ea59765cfc17cb57e2b6fcbf3e36a"} Jul 10 03:30:43.838699 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.838470 2585 scope.go:117] "RemoveContainer" containerID="ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053" Jul 10 03:30:43.851414 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.851395 2585 scope.go:117] "RemoveContainer" containerID="ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053" Jul 10 03:30:43.851715 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:30:43.851696 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053\": container with ID starting with ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053 not found: ID does not exist" containerID="ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053" Jul 10 03:30:43.851798 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.851727 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053"} err="failed to get container status \"ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053\": rpc error: code = NotFound desc = could not find container \"ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053\": container with ID starting with ce501a87d01f02b4b7c483f2f495b43945580a20eb1cf40c4c8c5398351d1053 not found: ID does not exist" Jul 10 03:30:43.864283 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.864259 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt"] Jul 10 03:30:43.867179 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:43.867155 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e7f792-65ff9d6fc-626qt"] Jul 10 03:30:44.112516 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:44.112439 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d89590e5-1f63-4db9-819b-541c11a39d5d" path="/var/lib/kubelet/pods/d89590e5-1f63-4db9-819b-541c11a39d5d/volumes" Jul 10 03:30:47.255338 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:47.255257 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:30:49.264542 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:49.264496 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:49.331593 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:49.331560 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:49.566574 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:49.566477 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:49.812772 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:49.812697 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:50.557117 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:50.557063 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:30:50.667454 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:30:50.667421 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:31:50.331908 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:50.331875 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:31:56.063824 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.063787 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr"] Jul 10 03:31:56.064253 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.064023 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" podUID="c7134417-ea8d-4722-a71c-f7162e3ea3bc" containerName="maas-api" containerID="cri-o://a87f843b033f24a7b757683c7dae7c00d37cf1fd3c62f794bd9b449a671dec28" gracePeriod=30 Jul 10 03:31:56.087133 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.087106 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:31:56.246965 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.246832 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7134417-ea8d-4722-a71c-f7162e3ea3bc" containerID="a87f843b033f24a7b757683c7dae7c00d37cf1fd3c62f794bd9b449a671dec28" exitCode=0 Jul 10 03:31:56.246965 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.246913 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" event={"ID":"c7134417-ea8d-4722-a71c-f7162e3ea3bc","Type":"ContainerDied","Data":"a87f843b033f24a7b757683c7dae7c00d37cf1fd3c62f794bd9b449a671dec28"} Jul 10 03:31:56.325899 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.325844 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:31:56.398700 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.398668 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:31:56.401742 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.401707 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7134417-ea8d-4722-a71c-f7162e3ea3bc-maas-api-tls\") pod \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " Jul 10 03:31:56.401884 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.401805 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srxgv\" (UniqueName: \"kubernetes.io/projected/c7134417-ea8d-4722-a71c-f7162e3ea3bc-kube-api-access-srxgv\") pod \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " Jul 10 03:31:56.401884 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.401854 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7134417-ea8d-4722-a71c-f7162e3ea3bc-openshift-service-ca\") pod \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\" (UID: \"c7134417-ea8d-4722-a71c-f7162e3ea3bc\") " Jul 10 03:31:56.402238 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.402215 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7134417-ea8d-4722-a71c-f7162e3ea3bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7134417-ea8d-4722-a71c-f7162e3ea3bc" (UID: "c7134417-ea8d-4722-a71c-f7162e3ea3bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:31:56.403709 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.403683 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7134417-ea8d-4722-a71c-f7162e3ea3bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7134417-ea8d-4722-a71c-f7162e3ea3bc" (UID: "c7134417-ea8d-4722-a71c-f7162e3ea3bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:31:56.403881 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.403854 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7134417-ea8d-4722-a71c-f7162e3ea3bc-kube-api-access-srxgv" (OuterVolumeSpecName: "kube-api-access-srxgv") pod "c7134417-ea8d-4722-a71c-f7162e3ea3bc" (UID: "c7134417-ea8d-4722-a71c-f7162e3ea3bc"). InnerVolumeSpecName "kube-api-access-srxgv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:31:56.503307 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.503271 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7134417-ea8d-4722-a71c-f7162e3ea3bc-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:31:56.503307 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.503302 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srxgv\" (UniqueName: \"kubernetes.io/projected/c7134417-ea8d-4722-a71c-f7162e3ea3bc-kube-api-access-srxgv\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:31:56.503307 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:56.503312 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7134417-ea8d-4722-a71c-f7162e3ea3bc-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:31:57.255182 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:57.255148 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" event={"ID":"c7134417-ea8d-4722-a71c-f7162e3ea3bc","Type":"ContainerDied","Data":"05e88c587596a5e0719e0b1432054bea9de64239daeb0d9a91a527ab3c07d8f9"} Jul 10 03:31:57.255600 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:57.255205 2585 scope.go:117] "RemoveContainer" containerID="a87f843b033f24a7b757683c7dae7c00d37cf1fd3c62f794bd9b449a671dec28" Jul 10 03:31:57.255600 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:57.255162 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr" Jul 10 03:31:57.279173 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:57.279143 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr"] Jul 10 03:31:57.281356 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:57.281334 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-48f01f-7b9b89c5c9-2pbjr"] Jul 10 03:31:58.112294 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:58.112263 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7134417-ea8d-4722-a71c-f7162e3ea3bc" path="/var/lib/kubelet/pods/c7134417-ea8d-4722-a71c-f7162e3ea3bc/volumes" Jul 10 03:31:59.544696 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:31:59.544661 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:32:01.874936 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:01.874904 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:02.001234 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:02.001196 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:02.196335 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:02.196295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:02.452450 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:02.452369 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:02.832410 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:02.832328 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:03.148971 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:03.148865 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:03.329212 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:03.329181 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:06.528863 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.528834 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:06.627285 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.627256 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:06.746177 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746148 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6"] Jul 10 03:32:06.746541 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746529 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7134417-ea8d-4722-a71c-f7162e3ea3bc" containerName="maas-api" Jul 10 03:32:06.746541 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746541 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7134417-ea8d-4722-a71c-f7162e3ea3bc" containerName="maas-api" Jul 10 03:32:06.746677 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746554 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb754d3b-b48e-4e5b-af4e-04439fe1d391" containerName="maas-api" Jul 10 03:32:06.746677 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746560 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb754d3b-b48e-4e5b-af4e-04439fe1d391" containerName="maas-api" Jul 10 03:32:06.746677 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746568 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89590e5-1f63-4db9-819b-541c11a39d5d" containerName="maas-api" Jul 10 03:32:06.746677 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746576 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89590e5-1f63-4db9-819b-541c11a39d5d" containerName="maas-api" Jul 10 03:32:06.746677 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746666 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7134417-ea8d-4722-a71c-f7162e3ea3bc" containerName="maas-api" Jul 10 03:32:06.746677 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746679 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89590e5-1f63-4db9-819b-541c11a39d5d" containerName="maas-api" Jul 10 03:32:06.746880 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.746689 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb754d3b-b48e-4e5b-af4e-04439fe1d391" containerName="maas-api" Jul 10 03:32:06.749857 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.749837 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.752402 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.752380 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1bdfdf-serving-cert\"" Jul 10 03:32:06.761738 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.761714 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6"] Jul 10 03:32:06.783205 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.783109 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:06.798777 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.798751 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.798927 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.798823 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677db978-a720-4bca-ba62-c179b32d7c30-openshift-service-ca\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.798927 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.798856 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfpfx\" (UniqueName: \"kubernetes.io/projected/677db978-a720-4bca-ba62-c179b32d7c30-kube-api-access-kfpfx\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.899980 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.899949 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677db978-a720-4bca-ba62-c179b32d7c30-openshift-service-ca\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.900190 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.899989 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfpfx\" (UniqueName: \"kubernetes.io/projected/677db978-a720-4bca-ba62-c179b32d7c30-kube-api-access-kfpfx\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.900190 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.900039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.900190 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:32:06.900186 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-serving-cert: secret "maas-api-e2e-api-a-1bdfdf-serving-cert" not found Jul 10 03:32:06.900363 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:32:06.900270 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls podName:677db978-a720-4bca-ba62-c179b32d7c30 nodeName:}" failed. No retries permitted until 2026-07-10 03:32:07.40024624 +0000 UTC m=+3134.109054775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls") pod "maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" (UID: "677db978-a720-4bca-ba62-c179b32d7c30") : secret "maas-api-e2e-api-a-1bdfdf-serving-cert" not found Jul 10 03:32:06.900703 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.900686 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677db978-a720-4bca-ba62-c179b32d7c30-openshift-service-ca\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:06.925762 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:06.925732 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfpfx\" (UniqueName: \"kubernetes.io/projected/677db978-a720-4bca-ba62-c179b32d7c30-kube-api-access-kfpfx\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:07.222773 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.222744 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:07.322069 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.322034 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:07.405441 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.405407 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:07.407826 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.407799 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls\") pod \"maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:07.419296 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.419271 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:07.660871 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.660779 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:07.804691 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.804601 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6"] Jul 10 03:32:07.814086 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:32:07.814049 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677db978_a720_4bca_ba62_c179b32d7c30.slice/crio-799e76e497181694b139aa0be6b311250cd4a806de920fd9b493da1ac10ee65b WatchSource:0}: Error finding container 799e76e497181694b139aa0be6b311250cd4a806de920fd9b493da1ac10ee65b: Status 404 returned error can't find the container with id 799e76e497181694b139aa0be6b311250cd4a806de920fd9b493da1ac10ee65b Jul 10 03:32:07.815478 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:07.815453 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 03:32:08.319615 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:08.319580 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" event={"ID":"677db978-a720-4bca-ba62-c179b32d7c30","Type":"ContainerStarted","Data":"799e76e497181694b139aa0be6b311250cd4a806de920fd9b493da1ac10ee65b"} Jul 10 03:32:09.326572 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:09.326535 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" event={"ID":"677db978-a720-4bca-ba62-c179b32d7c30","Type":"ContainerStarted","Data":"7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d"} Jul 10 03:32:09.326975 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:09.326606 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:09.345182 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:09.345134 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" podStartSLOduration=2.844226795 podStartE2EDuration="3.345118908s" totalImagesPullingTime="500.892113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 03:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:32:09.342499903 +0000 UTC m=+3136.051308452" watchObservedRunningTime="2026-07-10 03:32:09.345118908 +0000 UTC m=+3136.053927442" Jul 10 03:32:11.079948 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:11.079910 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:11.352587 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:11.352497 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:32:11.571269 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:11.571235 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:11.903402 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:11.903369 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:12.412873 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:12.412842 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:14.574794 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:14.574748 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:14.674793 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:14.674751 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:14.832392 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:14.832306 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:14.975586 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:14.975543 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:15.337661 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:15.337572 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:32:15.407276 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:15.407244 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:15.507765 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:15.507726 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:19.110431 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:19.110390 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:21.864859 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:21.864825 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:32:27.859108 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:27.859074 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:27.959172 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:27.959142 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:37.159358 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:37.159320 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:37.265733 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:37.265698 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:38.486890 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:38.486856 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:38.587560 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:38.587517 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:32:38.726170 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:32:38.726140 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:27.766569 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:27.766531 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:33.306846 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:33.306805 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6"] Jul 10 03:33:33.335415 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:33.335380 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:33.434989 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:33.434956 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:33.798103 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:33.798065 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" podUID="677db978-a720-4bca-ba62-c179b32d7c30" containerName="maas-api" containerID="cri-o://7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d" gracePeriod=30 Jul 10 03:33:34.051741 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.051674 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:33:34.108077 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.108039 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfpfx\" (UniqueName: \"kubernetes.io/projected/677db978-a720-4bca-ba62-c179b32d7c30-kube-api-access-kfpfx\") pod \"677db978-a720-4bca-ba62-c179b32d7c30\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " Jul 10 03:33:34.108265 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.108111 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls\") pod \"677db978-a720-4bca-ba62-c179b32d7c30\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " Jul 10 03:33:34.108265 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.108134 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677db978-a720-4bca-ba62-c179b32d7c30-openshift-service-ca\") pod \"677db978-a720-4bca-ba62-c179b32d7c30\" (UID: \"677db978-a720-4bca-ba62-c179b32d7c30\") " Jul 10 03:33:34.108572 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.108536 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677db978-a720-4bca-ba62-c179b32d7c30-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "677db978-a720-4bca-ba62-c179b32d7c30" (UID: "677db978-a720-4bca-ba62-c179b32d7c30"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 03:33:34.110344 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.110307 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "677db978-a720-4bca-ba62-c179b32d7c30" (UID: "677db978-a720-4bca-ba62-c179b32d7c30"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 03:33:34.110494 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.110398 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/677db978-a720-4bca-ba62-c179b32d7c30-kube-api-access-kfpfx" (OuterVolumeSpecName: "kube-api-access-kfpfx") pod "677db978-a720-4bca-ba62-c179b32d7c30" (UID: "677db978-a720-4bca-ba62-c179b32d7c30"). InnerVolumeSpecName "kube-api-access-kfpfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 03:33:34.208989 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.208949 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677db978-a720-4bca-ba62-c179b32d7c30-maas-api-tls\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:33:34.208989 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.208984 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677db978-a720-4bca-ba62-c179b32d7c30-openshift-service-ca\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:33:34.208989 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.208994 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfpfx\" (UniqueName: \"kubernetes.io/projected/677db978-a720-4bca-ba62-c179b32d7c30-kube-api-access-kfpfx\") on node \"ip-10-0-139-175.ec2.internal\" DevicePath \"\"" Jul 10 03:33:34.805826 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.805791 2585 generic.go:356] "Generic (PLEG): container finished" podID="677db978-a720-4bca-ba62-c179b32d7c30" containerID="7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d" exitCode=0 Jul 10 03:33:34.806301 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.805853 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" Jul 10 03:33:34.806301 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.805881 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" event={"ID":"677db978-a720-4bca-ba62-c179b32d7c30","Type":"ContainerDied","Data":"7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d"} Jul 10 03:33:34.806301 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.805945 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6" event={"ID":"677db978-a720-4bca-ba62-c179b32d7c30","Type":"ContainerDied","Data":"799e76e497181694b139aa0be6b311250cd4a806de920fd9b493da1ac10ee65b"} Jul 10 03:33:34.806301 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.805973 2585 scope.go:117] "RemoveContainer" containerID="7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d" Jul 10 03:33:34.815584 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.815561 2585 scope.go:117] "RemoveContainer" containerID="7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d" Jul 10 03:33:34.815895 ip-10-0-139-175 kubenswrapper[2585]: E0710 03:33:34.815875 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d\": container with ID starting with 7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d not found: ID does not exist" containerID="7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d" Jul 10 03:33:34.815969 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.815911 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d"} err="failed to get container status \"7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d\": rpc error: code = NotFound desc = could not find container \"7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d\": container with ID starting with 7e8f4687dc6c4ce481ff2a9798d73fd0bbf321d02f3621d3cae435233112a24d not found: ID does not exist" Jul 10 03:33:34.823018 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.822991 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6"] Jul 10 03:33:34.826554 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:34.826531 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1bdfdf-86c469667f-d5zf6"] Jul 10 03:33:36.112625 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:36.112587 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677db978-a720-4bca-ba62-c179b32d7c30" path="/var/lib/kubelet/pods/677db978-a720-4bca-ba62-c179b32d7c30/volumes" Jul 10 03:33:36.950520 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:36.950487 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:33:38.985216 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:38.985183 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:39.158579 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:39.158544 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:39.303640 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:39.303559 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:39.544325 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:39.544295 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:33:51.072608 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:33:51.072520 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:40.085023 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:40.084987 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:45.011994 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:45.011956 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:46.190874 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:46.190829 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:46.260523 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:46.260476 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:49.052178 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:49.052139 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:34:51.454680 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:51.454620 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:51.515914 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:51.515877 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:51.745582 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:51.745495 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:52.014986 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:52.014903 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:52.393942 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:52.393854 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:52.716948 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:52.716917 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:52.902435 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:52.902403 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:54.138831 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:54.138802 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:34:54.146730 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:54.146697 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:34:55.469366 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:55.469324 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:55.544148 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:55.544110 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:55.615545 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:55.615509 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:55.713319 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:55.713288 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:56.124746 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:56.124715 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:34:56.225690 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:34:56.225635 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:00.032758 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:00.032718 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:00.383056 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:00.382959 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:00.549935 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:00.549897 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:35:00.698843 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:00.698806 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:00.931699 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:00.931633 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:03.490642 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:03.490611 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:03.570117 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:03.570071 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:03.646166 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:03.646133 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:03.741231 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:03.741146 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:04.112306 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:04.112219 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:04.210175 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:04.210142 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:08.032255 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:08.032221 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:13.968160 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:13.968123 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:35:16.877247 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:16.877170 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:16.977873 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:16.977838 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:25.003268 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:25.003236 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:25.104014 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:25.103980 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:28.907034 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:28.907001 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:29.003017 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:29.002979 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:29.103113 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:29.103078 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:35:33.048116 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:35:33.048074 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:36:03.088112 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:36:03.088076 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:36:06.950407 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:36:06.950370 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:36:22.353132 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:36:22.353077 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:36:41.555901 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:36:41.555860 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:36:52.756111 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:36:52.756014 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:36:57.251130 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:36:57.251089 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:37:11.856625 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:37:11.856590 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:37:25.953674 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:37:25.949515 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:37:39.757599 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:37:39.757558 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:37:55.654374 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:37:55.654339 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:38:14.453568 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:14.453529 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:38:29.446831 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:29.446794 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:38:38.977344 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:38.977310 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:38:39.762460 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:39.762433 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:38:45.360838 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:45.360802 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:38:46.003921 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:46.003867 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:38:48.654818 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:38:48.654779 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:39:34.142073 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:39:34.142037 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:39:39.452530 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:39:39.452489 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:39:54.173266 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:39:54.173238 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:39:54.182295 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:39:54.182269 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:39:54.557115 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:39:54.557078 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:40:11.472495 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:11.472460 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:40:17.869815 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:17.869765 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:17.970153 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:17.970122 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:24.061169 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:24.061127 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:40:25.759765 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:25.759732 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:25.856407 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:25.856376 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:29.639343 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:29.639303 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:29.735577 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:29.735549 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:40.148258 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:40.148224 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:40:43.172387 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:43.172358 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:44.210447 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:44.210404 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:40:57.047485 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:40:57.047452 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:41:07.477032 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:07.476973 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:41:11.344609 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:11.344564 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:41:24.379328 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:24.379293 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:41:30.352319 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:30.352279 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:41:40.459168 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:40.459132 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:41:40.505562 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:40.505529 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:41:44.853965 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:41:44.853920 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:42:02.755865 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:42:02.755820 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:42:15.863089 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:42:15.863058 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:42:16.799195 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:42:16.799158 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:42:21.645953 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:42:21.645909 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:42:36.145927 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:42:36.145893 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:43:07.083490 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:43:07.083459 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:43:07.181725 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:43:07.181693 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:06.967927 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:06.967892 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:12.432126 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:12.432095 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:12.748267 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:12.748169 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:15.850973 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:15.850885 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:44:18.321691 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:18.321640 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:18.388792 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:18.388753 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:18.624154 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:18.624065 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:18.869613 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:18.869583 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:19.515107 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:19.515071 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:19.615606 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:19.615554 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:27.089150 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:27.089110 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:54.207728 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:54.207681 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:44:54.218619 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:54.218597 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:44:57.067610 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:57.067578 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:44:57.118435 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:44:57.118403 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:19.415079 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:19.415045 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:24.856355 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:24.856319 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:25.480413 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:25.480375 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:28.358171 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:28.358137 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-l2bkv"] Jul 10 03:45:30.383251 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:30.383219 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:30.487115 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:30.487079 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:31.028347 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:31.028316 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:31.080174 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:31.080147 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:31.327930 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:31.327840 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:31.579186 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:31.579104 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:37.307638 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:37.307597 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cz2zt_dcea5d17-55fb-42aa-bc6f-9d993ecebae0/manager/0.log" Jul 10 03:45:37.432968 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:37.432940 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f9ff5cc88-wlwgw_8d1a423b-56b2-41d2-95f0-ab20d745c594/manager/0.log" Jul 10 03:45:37.809964 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:37.809931 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-d72rh_670a16a3-4588-4d54-836e-307166d53f89/copy-manifests/0.log" Jul 10 03:45:37.822366 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:37.822334 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-d72rh_670a16a3-4588-4d54-836e-307166d53f89/manager/0.log" Jul 10 03:45:38.450581 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:38.450547 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b87df5b5f-zbrdq_ec2a51f8-8cae-4598-80c7-0dd7cf46dcb5/maas-api/0.log" Jul 10 03:45:39.642758 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.642725 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl_d0e83868-33a2-4f7d-8d43-6fcdac3bb079/pull/0.log" Jul 10 03:45:39.650123 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.650103 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl_d0e83868-33a2-4f7d-8d43-6fcdac3bb079/extract/0.log" Jul 10 03:45:39.656848 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.656822 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl_d0e83868-33a2-4f7d-8d43-6fcdac3bb079/util/0.log" Jul 10 03:45:39.773921 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.773895 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh_eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da/util/0.log" Jul 10 03:45:39.780152 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.780125 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh_eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da/pull/0.log" Jul 10 03:45:39.786114 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.786096 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh_eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da/extract/0.log" Jul 10 03:45:39.908064 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.907991 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf_564b90d2-f39c-48a6-b2e6-4c02527c0bf9/pull/0.log" Jul 10 03:45:39.913929 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.913909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf_564b90d2-f39c-48a6-b2e6-4c02527c0bf9/extract/0.log" Jul 10 03:45:39.919478 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:39.919461 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf_564b90d2-f39c-48a6-b2e6-4c02527c0bf9/util/0.log" Jul 10 03:45:40.044744 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:40.044719 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m_3c09e201-78a3-4db7-9613-d9d3ed0e9344/util/0.log" Jul 10 03:45:40.051403 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:40.051383 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m_3c09e201-78a3-4db7-9613-d9d3ed0e9344/pull/0.log" Jul 10 03:45:40.057581 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:40.057558 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m_3c09e201-78a3-4db7-9613-d9d3ed0e9344/extract/0.log" Jul 10 03:45:40.437362 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:40.437332 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wwvmf_f40b5f77-9a6e-43da-a040-938da2abdbb2/manager/0.log" Jul 10 03:45:40.813925 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:40.813831 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-74gvm_2ba19253-d03c-45d7-87b1-e3fecdfc3ad5/manager/0.log" Jul 10 03:45:40.934949 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:40.934920 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-l2bkv_7605e3f0-ecb3-46d9-bf33-4664f86e17f3/limitador/0.log" Jul 10 03:45:41.573301 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:41.573266 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w6wll_de76de0e-68e5-4522-8bf0-5f9dfb5ef663/discovery/0.log" Jul 10 03:45:41.826713 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:41.826592 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d56d99d47-xr2qn_cf17ac4b-09a5-4af0-b018-deb5b4c52b45/kube-auth-proxy/0.log" Jul 10 03:45:42.086112 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:42.086020 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-njkmk_64d09dd8-c14c-4453-946f-8b4838c3c28f/payload-pre-processing/0.log" Jul 10 03:45:42.222786 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:42.222760 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5thg8_dca6e4af-0ff6-4c18-904a-8b7f0307b109/payload-processing/0.log" Jul 10 03:45:42.836158 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:42.836134 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-n6nh8_c0336965-8245-43c5-84c8-1cbb27cbc1bd/main/0.log" Jul 10 03:45:43.210635 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:43.210604 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-v5txv_a441f833-ad77-43a4-943a-45406d9b7824/main/0.log" Jul 10 03:45:48.004885 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:48.004773 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:48.094807 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:48.094778 2585 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" limit=1024 Jul 10 03:45:50.919928 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:50.919883 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wd6fm_0991902f-cbfb-41d6-9399-519aac9b646b/global-pull-secret-syncer/0.log" Jul 10 03:45:50.969922 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:50.969887 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-857wz_c39ba390-03ea-466c-80c1-8dfdb19e32dd/konnectivity-agent/0.log" Jul 10 03:45:51.108870 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:51.108842 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-175.ec2.internal_fb0a5737ac1f577cebb69602803ae7c2/haproxy/0.log" Jul 10 03:45:55.570528 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.570497 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl_d0e83868-33a2-4f7d-8d43-6fcdac3bb079/extract/0.log" Jul 10 03:45:55.595554 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.595527 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl_d0e83868-33a2-4f7d-8d43-6fcdac3bb079/util/0.log" Jul 10 03:45:55.618435 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.618401 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d2qcl_d0e83868-33a2-4f7d-8d43-6fcdac3bb079/pull/0.log" Jul 10 03:45:55.646579 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.646550 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh_eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da/extract/0.log" Jul 10 03:45:55.670446 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.670421 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh_eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da/util/0.log" Jul 10 03:45:55.693405 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.693378 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05q5qh_eb3ef73a-b3fa-4faf-ba6f-e1f25315d9da/pull/0.log" Jul 10 03:45:55.721006 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.720981 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf_564b90d2-f39c-48a6-b2e6-4c02527c0bf9/extract/0.log" Jul 10 03:45:55.743818 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.743787 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf_564b90d2-f39c-48a6-b2e6-4c02527c0bf9/util/0.log" Jul 10 03:45:55.766580 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.766550 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pjrlf_564b90d2-f39c-48a6-b2e6-4c02527c0bf9/pull/0.log" Jul 10 03:45:55.795681 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.795631 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m_3c09e201-78a3-4db7-9613-d9d3ed0e9344/extract/0.log" Jul 10 03:45:55.820429 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.820403 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m_3c09e201-78a3-4db7-9613-d9d3ed0e9344/util/0.log" Jul 10 03:45:55.861283 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:55.861210 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dlg6m_3c09e201-78a3-4db7-9613-d9d3ed0e9344/pull/0.log" Jul 10 03:45:56.593520 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:56.593497 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-wwvmf_f40b5f77-9a6e-43da-a040-938da2abdbb2/manager/0.log" Jul 10 03:45:56.799007 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:56.798973 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-74gvm_2ba19253-d03c-45d7-87b1-e3fecdfc3ad5/manager/0.log" Jul 10 03:45:56.820623 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:56.820592 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-l2bkv_7605e3f0-ecb3-46d9-bf33-4664f86e17f3/limitador/0.log" Jul 10 03:45:58.923933 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:58.923903 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-878d8cd45-7jsqw_84511177-2df9-477f-9cc7-2420c1500205/metrics-server/0.log" Jul 10 03:45:58.953572 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:58.953540 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-fclkz_7f5d8651-57fe-4157-b6c0-5f051e302eb2/monitoring-plugin/0.log" Jul 10 03:45:59.154862 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.154832 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlsbc_b6059534-5d75-44fe-8288-fb02f5ec5e88/node-exporter/0.log" Jul 10 03:45:59.178613 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.178589 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlsbc_b6059534-5d75-44fe-8288-fb02f5ec5e88/kube-rbac-proxy/0.log" Jul 10 03:45:59.200671 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.200627 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qlsbc_b6059534-5d75-44fe-8288-fb02f5ec5e88/init-textfile/0.log" Jul 10 03:45:59.231311 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.231288 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-d9nwz_4e8bcddc-4610-4ab6-b030-eb1fbf49fd92/kube-rbac-proxy-main/0.log" Jul 10 03:45:59.256172 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.256148 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-d9nwz_4e8bcddc-4610-4ab6-b030-eb1fbf49fd92/kube-rbac-proxy-self/0.log" Jul 10 03:45:59.278173 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.278151 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-d9nwz_4e8bcddc-4610-4ab6-b030-eb1fbf49fd92/openshift-state-metrics/0.log" Jul 10 03:45:59.606738 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.606616 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bbd4bdcc5-28grk_f727ff78-9a77-4860-a4f9-779b745d6273/telemeter-client/0.log" Jul 10 03:45:59.633826 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.633784 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bbd4bdcc5-28grk_f727ff78-9a77-4860-a4f9-779b745d6273/reload/0.log" Jul 10 03:45:59.656716 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:45:59.656684 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bbd4bdcc5-28grk_f727ff78-9a77-4860-a4f9-779b745d6273/kube-rbac-proxy/0.log" Jul 10 03:46:01.378201 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.378165 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5"] Jul 10 03:46:01.378661 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.378626 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677db978-a720-4bca-ba62-c179b32d7c30" containerName="maas-api" Jul 10 03:46:01.378661 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.378660 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="677db978-a720-4bca-ba62-c179b32d7c30" containerName="maas-api" Jul 10 03:46:01.378765 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.378752 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="677db978-a720-4bca-ba62-c179b32d7c30" containerName="maas-api" Jul 10 03:46:01.382124 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.382107 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.384807 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.384780 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4zdlx\"/\"kube-root-ca.crt\"" Jul 10 03:46:01.384807 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.384795 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4zdlx\"/\"openshift-service-ca.crt\"" Jul 10 03:46:01.384968 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.384828 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4zdlx\"/\"default-dockercfg-6bsvz\"" Jul 10 03:46:01.392176 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.392152 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5"] Jul 10 03:46:01.467509 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.467470 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-sys\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.467773 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.467522 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-proc\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.467773 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.467644 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-lib-modules\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.467773 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.467725 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-podres\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.467773 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.467760 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hlvz\" (UniqueName: \"kubernetes.io/projected/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-kube-api-access-9hlvz\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569027 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.568984 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-sys\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-proc\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569101 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-lib-modules\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569119 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-podres\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569119 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-sys\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569164 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hlvz\" (UniqueName: \"kubernetes.io/projected/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-kube-api-access-9hlvz\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569181 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-proc\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569241 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569240 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-lib-modules\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.569574 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.569239 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-podres\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.576898 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.576875 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hlvz\" (UniqueName: \"kubernetes.io/projected/2d23863a-30b2-49c3-b18e-9f8101c4fd9f-kube-api-access-9hlvz\") pod \"perf-node-gather-daemonset-8vrz5\" (UID: \"2d23863a-30b2-49c3-b18e-9f8101c4fd9f\") " pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.641280 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.641194 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/1.log" Jul 10 03:46:01.646879 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.646856 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-xtwmx_8e54d5b6-b253-4343-a8b0-8c22d69bec95/console-operator/2.log" Jul 10 03:46:01.692382 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.692347 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.826228 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.826204 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5"] Jul 10 03:46:01.827224 ip-10-0-139-175 kubenswrapper[2585]: W0710 03:46:01.827196 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d23863a_30b2_49c3_b18e_9f8101c4fd9f.slice/crio-7860b20999cfe192c3275d01613a144780414fbcea0ccc16919f66af1e1d505a WatchSource:0}: Error finding container 7860b20999cfe192c3275d01613a144780414fbcea0ccc16919f66af1e1d505a: Status 404 returned error can't find the container with id 7860b20999cfe192c3275d01613a144780414fbcea0ccc16919f66af1e1d505a Jul 10 03:46:01.930337 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.930303 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" event={"ID":"2d23863a-30b2-49c3-b18e-9f8101c4fd9f","Type":"ContainerStarted","Data":"091890eb8dd2f37494584918956f35498d6f957cabaf5501d55c2a807b07c58b"} Jul 10 03:46:01.930584 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.930357 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" event={"ID":"2d23863a-30b2-49c3-b18e-9f8101c4fd9f","Type":"ContainerStarted","Data":"7860b20999cfe192c3275d01613a144780414fbcea0ccc16919f66af1e1d505a"} Jul 10 03:46:01.930584 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.930476 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:01.947523 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:01.947476 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" podStartSLOduration=0.947460874 podStartE2EDuration="947.460874ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 03:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 03:46:01.945066441 +0000 UTC m=+3968.653874977" watchObservedRunningTime="2026-07-10 03:46:01.947460874 +0000 UTC m=+3968.656269416" Jul 10 03:46:02.190269 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:02.190241 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-457qp_5f80bd45-9108-40f6-b232-82b2ad2e4b03/download-server/0.log" Jul 10 03:46:02.790698 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:02.790641 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-76lb8_2bda1c2d-6412-4f63-8f99-10cbf44fa771/volume-data-source-validator/0.log" Jul 10 03:46:03.826975 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:03.826949 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4w9w5_33303bd9-1b00-4884-a4bb-028e5620fafe/dns/0.log" Jul 10 03:46:03.850766 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:03.850744 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4w9w5_33303bd9-1b00-4884-a4bb-028e5620fafe/kube-rbac-proxy/0.log" Jul 10 03:46:04.037770 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:04.037733 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k6qd5_fb1de517-e278-4495-bc0e-02c07d5964d3/dns-node-resolver/0.log" Jul 10 03:46:04.656807 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:04.656774 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5d5499fc78-n2xpq_b3e24133-eb82-45ae-a1a0-534d64a4528d/registry/0.log" Jul 10 03:46:04.683222 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:04.683197 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-69ntc_41878b22-23b8-43fc-b63d-e753fd7c5b5c/node-ca/0.log" Jul 10 03:46:05.798567 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:05.798533 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w6wll_de76de0e-68e5-4522-8bf0-5f9dfb5ef663/discovery/0.log" Jul 10 03:46:05.846190 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:05.846153 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d56d99d47-xr2qn_cf17ac4b-09a5-4af0-b018-deb5b4c52b45/kube-auth-proxy/0.log" Jul 10 03:46:05.942284 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:05.942253 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-njkmk_64d09dd8-c14c-4453-946f-8b4838c3c28f/payload-pre-processing/0.log" Jul 10 03:46:05.971478 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:05.971441 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5thg8_dca6e4af-0ff6-4c18-904a-8b7f0307b109/payload-processing/0.log" Jul 10 03:46:06.791230 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:06.791205 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jktcm_f28708fe-6f76-4cc0-b5fe-a9e15c62ecdd/serve-healthcheck-canary/0.log" Jul 10 03:46:07.291128 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:07.291099 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7vr8g_05fdce4d-a91e-46e1-9326-0c151298684b/kube-rbac-proxy/0.log" Jul 10 03:46:07.312604 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:07.312572 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7vr8g_05fdce4d-a91e-46e1-9326-0c151298684b/exporter/0.log" Jul 10 03:46:07.334357 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:07.334325 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7vr8g_05fdce4d-a91e-46e1-9326-0c151298684b/extractor/0.log" Jul 10 03:46:07.945822 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:07.945790 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4zdlx/perf-node-gather-daemonset-8vrz5" Jul 10 03:46:09.756939 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:09.756909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cz2zt_dcea5d17-55fb-42aa-bc6f-9d993ecebae0/manager/0.log" Jul 10 03:46:09.863968 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:09.863932 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f9ff5cc88-wlwgw_8d1a423b-56b2-41d2-95f0-ab20d745c594/manager/0.log" Jul 10 03:46:10.067642 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:10.067610 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-d72rh_670a16a3-4588-4d54-836e-307166d53f89/manager/0.log" Jul 10 03:46:10.088012 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:10.087987 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68647b69dd-d72rh_670a16a3-4588-4d54-836e-307166d53f89/copy-manifests/0.log" Jul 10 03:46:11.723516 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:11.723487 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jfhsj_8bb7d3f1-20ec-4af3-a668-59d5176fa1e4/openshift-lws-operator/0.log" Jul 10 03:46:12.271264 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:12.271230 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w6wll_de76de0e-68e5-4522-8bf0-5f9dfb5ef663/discovery/0.log" Jul 10 03:46:12.299485 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:12.299463 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d56d99d47-xr2qn_cf17ac4b-09a5-4af0-b018-deb5b4c52b45/kube-auth-proxy/0.log" Jul 10 03:46:12.372214 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:12.372182 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-njkmk_64d09dd8-c14c-4453-946f-8b4838c3c28f/payload-pre-processing/0.log" Jul 10 03:46:12.393113 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:12.393085 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5thg8_dca6e4af-0ff6-4c18-904a-8b7f0307b109/payload-processing/0.log" Jul 10 03:46:13.803854 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:13.803817 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-w6wll_de76de0e-68e5-4522-8bf0-5f9dfb5ef663/discovery/0.log" Jul 10 03:46:13.833208 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:13.833181 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d56d99d47-xr2qn_cf17ac4b-09a5-4af0-b018-deb5b4c52b45/kube-auth-proxy/0.log" Jul 10 03:46:13.908239 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:13.908209 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-njkmk_64d09dd8-c14c-4453-946f-8b4838c3c28f/payload-pre-processing/0.log" Jul 10 03:46:13.930105 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:13.930072 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5thg8_dca6e4af-0ff6-4c18-904a-8b7f0307b109/payload-processing/0.log" Jul 10 03:46:16.854635 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:16.854601 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-ckf8c_f0c93225-a245-4389-8f5b-86810e3ac838/migrator/0.log" Jul 10 03:46:16.874823 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:16.874793 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-ckf8c_f0c93225-a245-4389-8f5b-86810e3ac838/graceful-termination/0.log" Jul 10 03:46:17.230396 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:17.230368 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rp59d_b1972ee0-9c38-4c7d-be80-4a6cebda905f/kube-storage-version-migrator-operator/1.log" Jul 10 03:46:17.231120 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:17.231089 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rp59d_b1972ee0-9c38-4c7d-be80-4a6cebda905f/kube-storage-version-migrator-operator/0.log" Jul 10 03:46:18.242711 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.242683 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/kube-multus-additional-cni-plugins/0.log" Jul 10 03:46:18.264601 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.264569 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/egress-router-binary-copy/0.log" Jul 10 03:46:18.286257 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.286227 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/cni-plugins/0.log" Jul 10 03:46:18.309100 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.309075 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/bond-cni-plugin/0.log" Jul 10 03:46:18.330619 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.330595 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/routeoverride-cni/0.log" Jul 10 03:46:18.352474 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.352439 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/whereabouts-cni-bincopy/0.log" Jul 10 03:46:18.373162 ip-10-0-139-175 kubenswrapper[2585]: I0710 03:46:18.373138 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2kxtv_3689d3fd-6681-4c30-aa1f-53d31bae8186/whereabouts-cni/0.log"