Aug 05 17:17:27.731771 ip-10-0-143-58 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Aug 05 17:17:27.731781 ip-10-0-143-58 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Aug 05 17:17:27.731788 ip-10-0-143-58 systemd[1]: kubelet.service: Failed with result 'resources'. Aug 05 17:17:27.732007 ip-10-0-143-58 systemd[1]: Failed to start Kubernetes Kubelet. Aug 05 17:17:37.777522 ip-10-0-143-58 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Aug 05 17:17:37.777536 ip-10-0-143-58 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3dd882dafad847db9cb1798899faa294 -- Aug 05 17:19:22.495319 ip-10-0-143-58 systemd[1]: Starting Kubernetes Kubelet... Aug 05 17:19:22.840496 ip-10-0-143-58 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 05 17:19:22.840496 ip-10-0-143-58 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 05 17:19:22.840496 ip-10-0-143-58 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 05 17:19:22.840496 ip-10-0-143-58 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 05 17:19:22.840496 ip-10-0-143-58 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 05 17:19:22.841287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.841204 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 05 17:19:22.844572 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844556 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 17:19:22.844572 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844571 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844577 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844582 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844585 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844588 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844591 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844600 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844604 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844607 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844610 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844612 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844615 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844618 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844621 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844623 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844626 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844629 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844631 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844634 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844637 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 17:19:22.844636 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844639 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844642 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844645 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844648 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844651 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844653 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844655 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844658 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844660 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844663 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844665 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844668 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844670 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844673 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844676 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844678 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844681 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844683 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844686 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844688 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 17:19:22.845122 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844691 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844693 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844695 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844698 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844700 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844703 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844705 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844708 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844710 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844713 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844715 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844717 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844719 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844722 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844726 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844728 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844731 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844733 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844736 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844738 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 17:19:22.845589 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844741 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844743 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844746 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844748 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844750 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844753 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844755 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844759 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844762 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844765 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844770 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844773 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844775 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844778 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844780 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844782 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844785 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844788 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844790 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 17:19:22.846072 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844792 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844795 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844798 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844801 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844803 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844806 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844808 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844811 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844814 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844816 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844819 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844821 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844823 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844826 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844828 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844831 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844833 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844836 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.844838 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845536 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 17:19:22.846533 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845544 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845548 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845550 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845553 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845556 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845559 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845562 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845564 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845567 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845569 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845572 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845574 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845577 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845579 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845582 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845584 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845587 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845589 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845592 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845595 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 17:19:22.847116 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845598 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845601 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845604 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845606 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845610 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845614 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845617 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845620 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845623 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845625 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845628 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845630 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845633 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845635 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845638 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845640 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845644 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845648 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845651 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 17:19:22.847603 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845654 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845657 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845660 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845663 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845665 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845668 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845672 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845674 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845677 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845680 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845682 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845684 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845687 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845690 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845694 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845697 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845700 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845702 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845705 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 17:19:22.848066 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845708 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845710 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845713 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845716 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845718 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845721 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845724 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845726 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845729 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845731 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845734 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845737 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845740 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845742 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845745 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845747 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845750 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845752 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845754 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845756 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 17:19:22.848539 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845759 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845762 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845764 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845769 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845772 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845774 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845776 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845779 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845781 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845784 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845786 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845790 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845792 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845795 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845797 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845800 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845802 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845805 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845808 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.845811 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 17:19:22.849007 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846612 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846618 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846622 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846626 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846630 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846633 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846636 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846638 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846641 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846643 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846646 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846649 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846652 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846654 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846657 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846662 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846664 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846667 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846669 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846672 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 17:19:22.849501 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846674 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846677 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846679 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846682 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846685 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846687 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846689 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846692 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846694 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846697 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846699 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846702 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846704 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846707 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846709 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846712 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846715 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846719 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846723 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 17:19:22.850001 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846726 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846728 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846731 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846733 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846736 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846738 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846741 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846744 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846748 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846750 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846753 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846755 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846758 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846760 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846762 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846765 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846767 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846769 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846772 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846774 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 17:19:22.850478 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846777 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846779 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846781 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846784 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846787 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846789 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846793 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846796 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846799 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846802 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846805 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846809 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846812 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846815 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846819 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846821 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846824 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846827 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846829 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 17:19:22.850953 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846832 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846836 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846838 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846841 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846843 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846846 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846848 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846851 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846853 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846856 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846858 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846860 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846863 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846865 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846868 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846870 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846873 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846876 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846879 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846881 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.846884 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 17:19:22.851482 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846957 2575 flags.go:64] FLAG: --address="0.0.0.0" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846967 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846975 2575 flags.go:64] FLAG: --anonymous-auth="true" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846981 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846989 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846992 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.846996 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847000 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847004 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847007 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847010 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847013 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847018 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847021 2575 flags.go:64] FLAG: --cgroup-root="" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847024 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847026 2575 flags.go:64] FLAG: --client-ca-file="" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847029 2575 flags.go:64] FLAG: --cloud-provider="external" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847032 2575 flags.go:64] FLAG: --cluster-dns="[]" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847036 2575 flags.go:64] FLAG: --cluster-domain="" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847039 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847042 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847044 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847059 2575 flags.go:64] FLAG: --container-log-max-files="5" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847063 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 05 17:19:22.851980 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847067 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847070 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847073 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847076 2575 flags.go:64] FLAG: --contention-profiling="false" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847079 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847081 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847085 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847087 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847091 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847094 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847097 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847100 2575 flags.go:64] FLAG: --enable-load-reader="false" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847102 2575 flags.go:64] FLAG: --enable-server="true" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847106 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847110 2575 flags.go:64] FLAG: --event-burst="100" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847113 2575 flags.go:64] FLAG: --event-qps="50" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847116 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847119 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847121 2575 flags.go:64] FLAG: --eviction-hard="" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847125 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847129 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847132 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847135 2575 flags.go:64] FLAG: --eviction-soft="" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847138 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847141 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 05 17:19:22.852556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847143 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847146 2575 flags.go:64] FLAG: --experimental-mounter-path="" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847149 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847152 2575 flags.go:64] FLAG: --fail-swap-on="true" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847155 2575 flags.go:64] FLAG: --feature-gates="" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847159 2575 flags.go:64] FLAG: --file-check-frequency="20s" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847161 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847164 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847168 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847171 2575 flags.go:64] FLAG: --healthz-port="10248" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847174 2575 flags.go:64] FLAG: --help="false" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847177 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-143-58.ec2.internal" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847180 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847182 2575 flags.go:64] FLAG: --http-check-frequency="20s" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847186 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847189 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847193 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847195 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847198 2575 flags.go:64] FLAG: --image-service-endpoint="" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847201 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847204 2575 flags.go:64] FLAG: --kube-api-burst="100" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847207 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847211 2575 flags.go:64] FLAG: --kube-api-qps="50" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847213 2575 flags.go:64] FLAG: --kube-reserved="" Aug 05 17:19:22.853213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847216 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847219 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847222 2575 flags.go:64] FLAG: --kubelet-cgroups="" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847225 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847228 2575 flags.go:64] FLAG: --lock-file="" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847231 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847233 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847236 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847241 2575 flags.go:64] FLAG: --log-json-split-stream="false" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847244 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847246 2575 flags.go:64] FLAG: --log-text-split-stream="false" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847249 2575 flags.go:64] FLAG: --logging-format="text" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847253 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847256 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847259 2575 flags.go:64] FLAG: --manifest-url="" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847262 2575 flags.go:64] FLAG: --manifest-url-header="" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847266 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847269 2575 flags.go:64] FLAG: --max-open-files="1000000" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847273 2575 flags.go:64] FLAG: --max-pods="110" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847276 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847279 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847281 2575 flags.go:64] FLAG: --memory-manager-policy="None" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847284 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847287 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847290 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 05 17:19:22.853814 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847293 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847299 2575 flags.go:64] FLAG: --node-status-max-images="50" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847302 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847305 2575 flags.go:64] FLAG: --oom-score-adj="-999" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847308 2575 flags.go:64] FLAG: --pod-cidr="" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847311 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847316 2575 flags.go:64] FLAG: --pod-manifest-path="" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847319 2575 flags.go:64] FLAG: --pod-max-pids="-1" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847322 2575 flags.go:64] FLAG: --pods-per-core="0" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847325 2575 flags.go:64] FLAG: --port="10250" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847328 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847331 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0057c9b888163eefc" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847334 2575 flags.go:64] FLAG: --qos-reserved="" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847337 2575 flags.go:64] FLAG: --read-only-port="10255" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847347 2575 flags.go:64] FLAG: --register-node="true" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847350 2575 flags.go:64] FLAG: --register-with-taints="" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847353 2575 flags.go:64] FLAG: --registry-burst="10" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847356 2575 flags.go:64] FLAG: --registry-qps="5" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847359 2575 flags.go:64] FLAG: --reserved-cpus="" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847362 2575 flags.go:64] FLAG: --reserved-memory="" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847367 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847370 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847373 2575 flags.go:64] FLAG: --rotate-certificates="false" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847376 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847378 2575 flags.go:64] FLAG: --runonce="false" Aug 05 17:19:22.854422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847381 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847384 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847387 2575 flags.go:64] FLAG: --seccomp-default="false" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847390 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847393 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847396 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847399 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847402 2575 flags.go:64] FLAG: --storage-driver-password="root" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847404 2575 flags.go:64] FLAG: --storage-driver-secure="false" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847407 2575 flags.go:64] FLAG: --storage-driver-table="stats" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847410 2575 flags.go:64] FLAG: --storage-driver-user="root" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847412 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847415 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847419 2575 flags.go:64] FLAG: --system-cgroups="" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847421 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847426 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847434 2575 flags.go:64] FLAG: --tls-cert-file="" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847437 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847443 2575 flags.go:64] FLAG: --tls-min-version="" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847447 2575 flags.go:64] FLAG: --tls-private-key-file="" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847449 2575 flags.go:64] FLAG: --topology-manager-policy="none" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847452 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847455 2575 flags.go:64] FLAG: --topology-manager-scope="container" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847457 2575 flags.go:64] FLAG: --v="2" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847461 2575 flags.go:64] FLAG: --version="false" Aug 05 17:19:22.855047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847465 2575 flags.go:64] FLAG: --vmodule="" Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847468 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847473 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847590 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847595 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847599 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847602 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847605 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847608 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847611 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847614 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847617 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847619 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847622 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847624 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847627 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847630 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847633 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847635 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 17:19:22.855647 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847638 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847640 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847648 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847651 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847654 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847656 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847659 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847661 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847664 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847666 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847668 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847671 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847673 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847676 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847678 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847682 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847684 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847687 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847689 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847691 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847694 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 17:19:22.856115 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847696 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847699 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847701 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847704 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847706 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847709 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847711 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847715 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847718 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847721 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847723 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847726 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847729 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847731 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847734 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847736 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847741 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847743 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847746 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 17:19:22.856605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847748 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847751 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847753 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847757 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847760 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847763 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847765 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847769 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847771 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847774 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847776 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847779 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847781 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847784 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847786 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847789 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847791 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847794 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847796 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 17:19:22.857262 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847799 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847802 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847804 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847806 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847809 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847811 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847814 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847816 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847819 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847822 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847825 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847828 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847830 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847833 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847835 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847837 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847840 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847842 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847845 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847847 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 17:19:22.857927 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847851 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847854 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847856 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.847859 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.847864 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.854673 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.854686 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.854703 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.854712 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854770 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854776 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854779 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854782 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854786 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854789 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 17:19:22.858423 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854791 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854794 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854796 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854799 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854801 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854804 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854807 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854809 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854812 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854814 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854818 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854822 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854826 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854828 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854831 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854833 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854836 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854838 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854841 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854844 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 17:19:22.858794 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854846 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854849 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854851 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854855 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854858 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854861 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854863 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854866 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854868 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854871 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854874 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854876 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854879 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854881 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854884 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854888 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854891 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854895 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854897 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854900 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 17:19:22.859303 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854903 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854906 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854909 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854911 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854915 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854917 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854920 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854923 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854926 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854928 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854931 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854933 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854935 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854938 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854941 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854943 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854946 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854949 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854951 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854953 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 17:19:22.859786 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854956 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854959 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854961 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854965 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854968 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854971 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854974 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854976 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854979 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854982 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854986 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854989 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854991 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854994 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854996 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.854999 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855001 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855004 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855006 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 17:19:22.860281 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855009 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855011 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855014 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855017 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855019 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855022 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855025 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855027 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855030 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855032 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855036 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855038 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855041 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855043 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.855062 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 05 17:19:22.860727 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855182 2575 feature_gate.go:328] unrecognized feature gate: Example Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855186 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855189 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855192 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855194 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855197 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855199 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855201 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855204 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855206 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855209 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855211 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855213 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855216 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855219 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855223 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855225 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855228 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855231 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 05 17:19:22.861119 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855233 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855235 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855238 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855241 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855243 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855245 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855248 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855251 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855253 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855256 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855258 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855260 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855263 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855265 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855268 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855270 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855273 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855275 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855277 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855280 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 05 17:19:22.861574 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855282 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855285 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855287 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855289 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855292 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855294 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855296 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855299 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855303 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855306 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855309 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855311 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855314 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855316 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855320 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855323 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855326 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855329 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855332 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 05 17:19:22.862040 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855336 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855345 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855348 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855352 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855355 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855358 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855360 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855363 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855365 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855367 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855370 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855372 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855375 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855377 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855380 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855382 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855384 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855387 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855389 2575 feature_gate.go:328] unrecognized feature gate: Example2 Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855392 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 05 17:19:22.862503 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855394 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855397 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855400 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855402 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855405 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855407 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855409 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855412 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855414 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855417 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855419 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855422 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855425 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855428 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855430 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855432 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855435 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855438 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855441 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855443 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 05 17:19:22.862971 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:22.855446 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 05 17:19:22.863558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.855451 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 05 17:19:22.863558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.855544 2575 server.go:962] "Client rotation is on, will bootstrap in background" Aug 05 17:19:22.863558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.860669 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 05 17:19:22.863558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.861922 2575 server.go:1019] "Starting client certificate rotation" Aug 05 17:19:22.863558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.862017 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 05 17:19:22.863558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.862062 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 05 17:19:22.864117 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.864104 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 05 17:19:22.866025 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.866013 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 05 17:19:22.873990 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.873973 2575 log.go:25] "Validated CRI v1 runtime API" Aug 05 17:19:22.878319 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.878301 2575 log.go:25] "Validated CRI v1 image API" Aug 05 17:19:22.880415 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.880392 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 05 17:19:22.882523 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.882503 2575 fs.go:135] Filesystem UUIDs: map[73cd1311-1ccf-4cd2-9483-ce0ec2de1f48:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f3dd6bb5-3a5a-4ed7-ba60-60e863e20b25:/dev/nvme0n1p4] Aug 05 17:19:22.882605 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.882521 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 05 17:19:22.887882 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.887773 2575 manager.go:216] Machine: {Timestamp:2026-08-05 17:19:22.886646404 +0000 UTC m=+0.287875210 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203587 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22c8324950055d94166cc8c0318dca SystemUUID:ec22c832-4950-055d-9416-6cc8c0318dca BootID:3dd882da-fad8-47db-9cb1-798899faa294 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7f:b8:70:5c:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7f:b8:70:5c:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:a9:1c:2a:98:d3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 05 17:19:22.887882 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.887873 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 05 17:19:22.888040 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.888013 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 05 17:19:22.890224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.890209 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 05 17:19:22.891027 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.890999 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 05 17:19:22.891170 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.891032 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-58.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 05 17:19:22.892225 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.892213 2575 topology_manager.go:138] "Creating topology manager with none policy" Aug 05 17:19:22.892258 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.892229 2575 container_manager_linux.go:309] "Creating device plugin manager" Aug 05 17:19:22.892258 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.892237 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 05 17:19:22.892806 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.892796 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 05 17:19:22.893807 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.893798 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 05 17:19:22.898243 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898231 2575 state_mem.go:36] "Initialized new in-memory state store" Aug 05 17:19:22.898370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898360 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 05 17:19:22.898407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898390 2575 kubelet.go:486] "Attempting to sync node with API server" Aug 05 17:19:22.898407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898400 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 05 17:19:22.898474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898413 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 05 17:19:22.898474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898422 2575 kubelet.go:398] "Adding apiserver pod source" Aug 05 17:19:22.898474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.898430 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 05 17:19:22.899047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.899031 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 05 17:19:22.899047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.899066 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 05 17:19:22.902594 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.902575 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 05 17:19:22.904084 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.904070 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 05 17:19:22.904142 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.904100 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 05 17:19:22.905237 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905226 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 05 17:19:22.905271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905242 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 05 17:19:22.905271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905248 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 05 17:19:22.905271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905253 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 05 17:19:22.905271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905261 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 05 17:19:22.905271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905266 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 05 17:19:22.905396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905274 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 05 17:19:22.905396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905279 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 05 17:19:22.905396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905285 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 05 17:19:22.905396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905290 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 05 17:19:22.905396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905299 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 05 17:19:22.905396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.905303 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 05 17:19:22.906130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.906121 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 05 17:19:22.906164 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.906131 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 05 17:19:22.908981 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.908965 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-58.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 05 17:19:22.909102 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.909084 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 05 17:19:22.909207 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.909189 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 05 17:19:22.909483 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.909472 2575 server.go:1268] "Started kubelet" Aug 05 17:19:22.909618 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.909564 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 05 17:19:22.909740 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.909640 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 05 17:19:22.909740 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.909592 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 05 17:19:22.910317 ip-10-0-143-58 systemd[1]: Started Kubernetes Kubelet. Aug 05 17:19:22.911008 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.910840 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 05 17:19:22.912383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.912370 2575 server.go:310] "Adding debug handlers to kubelet server" Aug 05 17:19:22.913649 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.913628 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 05 17:19:22.915788 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.914935 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-58.ec2.internal.18c8f86487b259e8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-58.ec2.internal,UID:ip-10-0-143-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-58.ec2.internal,},FirstTimestamp:2026-08-05 17:19:22.909432296 +0000 UTC m=+0.310661102,LastTimestamp:2026-08-05 17:19:22.909432296 +0000 UTC m=+0.310661102,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-58.ec2.internal,}" Aug 05 17:19:22.916816 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.916799 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 05 17:19:22.916816 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.916804 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 05 17:19:22.916959 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.916884 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:22.917393 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917378 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 05 17:19:22.917478 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917378 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 05 17:19:22.917478 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917472 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 05 17:19:22.917648 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917620 2575 reconstruct.go:97] "Volume reconstruction finished" Aug 05 17:19:22.917648 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917645 2575 reconciler.go:29] "Reconciler: start to sync state" Aug 05 17:19:22.917938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917924 2575 factory.go:153] Registering CRI-O factory Aug 05 17:19:22.918004 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.917964 2575 factory.go:223] Registration of the crio container factory successfully Aug 05 17:19:22.918065 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.918030 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 05 17:19:22.918065 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.918044 2575 factory.go:55] Registering systemd factory Aug 05 17:19:22.918065 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.918065 2575 factory.go:223] Registration of the systemd container factory successfully Aug 05 17:19:22.918226 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.918088 2575 factory.go:103] Registering Raw factory Aug 05 17:19:22.918226 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.918097 2575 manager.go:1196] Started watching for new ooms in manager Aug 05 17:19:22.919226 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.919208 2575 manager.go:349] Starting recovery of all containers Aug 05 17:19:22.929302 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.929268 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 05 17:19:22.929481 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.929457 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 05 17:19:22.930775 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.930757 2575 manager.go:354] Recovery completed Aug 05 17:19:22.935710 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.935698 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:22.937646 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.937623 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:22.937719 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.937658 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:22.937719 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.937669 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:22.938231 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938188 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 05 17:19:22.938231 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938199 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 05 17:19:22.938231 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938215 2575 state_mem.go:36] "Initialized new in-memory state store" Aug 05 17:19:22.938333 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938232 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:22.938919 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938906 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:22.938958 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938931 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:22.938958 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.938941 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:22.940232 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.940219 2575 policy_none.go:49] "None policy: Start" Aug 05 17:19:22.940289 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.940237 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 05 17:19:22.940289 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.940250 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 05 17:19:22.942063 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.942041 2575 policy_none.go:47] "Start" Aug 05 17:19:22.943826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.943806 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vfs56" Aug 05 17:19:22.949392 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.949322 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-58.ec2.internal.18c8f8648960db8d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-58.ec2.internal,UID:ip-10-0-143-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-58.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-58.ec2.internal,},FirstTimestamp:2026-08-05 17:19:22.937645965 +0000 UTC m=+0.338874772,LastTimestamp:2026-08-05 17:19:22.937645965 +0000 UTC m=+0.338874772,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-58.ec2.internal,}" Aug 05 17:19:22.952533 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.952515 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vfs56" Aug 05 17:19:22.958719 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.958638 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-58.ec2.internal.18c8f86489611ed2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-58.ec2.internal,UID:ip-10-0-143-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-58.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-58.ec2.internal,},FirstTimestamp:2026-08-05 17:19:22.937663186 +0000 UTC m=+0.338891991,LastTimestamp:2026-08-05 17:19:22.937663186 +0000 UTC m=+0.338891991,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-58.ec2.internal,}" Aug 05 17:19:22.975751 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.975632 2575 manager.go:337] "Starting Device Plugin manager" Aug 05 17:19:22.975826 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.975765 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 05 17:19:22.975826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.975778 2575 server.go:87] "Starting device plugin registration server" Aug 05 17:19:22.975995 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.975983 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 05 17:19:22.976040 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.975998 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 05 17:19:22.976119 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.976102 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 05 17:19:22.976216 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.976191 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 05 17:19:22.976216 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:22.976202 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 05 17:19:22.976668 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.976639 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 05 17:19:22.976760 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:22.976688 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.018535 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.018492 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 05 17:19:23.024986 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.019582 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 05 17:19:23.024986 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.019621 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 05 17:19:23.024986 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.019647 2575 kubelet.go:2448] "Starting kubelet main sync loop" Aug 05 17:19:23.024986 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.019685 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 05 17:19:23.024986 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.021840 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 17:19:23.076776 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.076753 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:23.077635 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.077615 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:23.077736 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.077649 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:23.077736 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.077662 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:23.077736 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.077691 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.087238 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.087217 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.087328 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.087242 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-58.ec2.internal\": node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.115065 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.114968 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.120200 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.120184 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal"] Aug 05 17:19:23.120250 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.120236 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:23.120928 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.120911 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:23.121004 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.120937 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:23.121004 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.120961 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:23.123220 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.123208 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:23.123330 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.123317 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.123382 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.123353 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:23.125809 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.125788 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:23.125905 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.125821 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:23.125905 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.125836 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:23.125905 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.125846 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:23.125905 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.125869 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:23.125905 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.125879 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:23.128113 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.128098 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.128192 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.128125 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 05 17:19:23.128738 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.128724 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientMemory" Aug 05 17:19:23.128826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.128747 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasNoDiskPressure" Aug 05 17:19:23.128826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.128758 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeHasSufficientPID" Aug 05 17:19:23.148633 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.148613 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-58.ec2.internal\" not found" node="ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.152787 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.152771 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-58.ec2.internal\" not found" node="ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.215678 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.215659 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.220007 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.219987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/025d669be6d578eb1f9fa3eb786bf236-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal\" (UID: \"025d669be6d578eb1f9fa3eb786bf236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.220087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.220014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47f3b5ee5e09766822cedb87a0da628d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-58.ec2.internal\" (UID: \"47f3b5ee5e09766822cedb87a0da628d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.220087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.220030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/025d669be6d578eb1f9fa3eb786bf236-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal\" (UID: \"025d669be6d578eb1f9fa3eb786bf236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.316271 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.316232 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.320583 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.320569 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/025d669be6d578eb1f9fa3eb786bf236-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal\" (UID: \"025d669be6d578eb1f9fa3eb786bf236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.320633 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.320592 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/025d669be6d578eb1f9fa3eb786bf236-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal\" (UID: \"025d669be6d578eb1f9fa3eb786bf236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.320633 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.320608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47f3b5ee5e09766822cedb87a0da628d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-58.ec2.internal\" (UID: \"47f3b5ee5e09766822cedb87a0da628d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.320703 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.320648 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47f3b5ee5e09766822cedb87a0da628d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-58.ec2.internal\" (UID: \"47f3b5ee5e09766822cedb87a0da628d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.320703 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.320678 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/025d669be6d578eb1f9fa3eb786bf236-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal\" (UID: \"025d669be6d578eb1f9fa3eb786bf236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.320763 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.320679 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/025d669be6d578eb1f9fa3eb786bf236-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal\" (UID: \"025d669be6d578eb1f9fa3eb786bf236\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.417231 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.417153 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.453250 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.453225 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.455831 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.455813 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.518091 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.518046 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.622600 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.622567 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.722699 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.722629 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.749070 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.749046 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 17:19:23.823592 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.823566 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-58.ec2.internal\" not found" Aug 05 17:19:23.861543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.861514 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 05 17:19:23.861981 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.861693 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 05 17:19:23.861981 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.861712 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 05 17:19:23.878585 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.878565 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 17:19:23.900576 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.900557 2575 apiserver.go:52] "Watching apiserver" Aug 05 17:19:23.911025 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.911003 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 05 17:19:23.911470 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.911450 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-9799r","openshift-multus/network-metrics-daemon-2m4bd","openshift-network-diagnostics/network-check-target-sgrxg","openshift-network-operator/iptables-alerter-49vv7","openshift-ovn-kubernetes/ovnkube-node-d2f2g","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf","openshift-cluster-node-tuning-operator/tuned-kplbj","openshift-dns/node-resolver-9gz9l","openshift-image-registry/node-ca-9cf2h","openshift-multus/multus-additional-cni-plugins-9mv62","kube-system/global-pull-secret-syncer-dpb7v","kube-system/konnectivity-agent-ffknk"] Aug 05 17:19:23.914148 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.914126 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9799r" Aug 05 17:19:23.916243 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.916222 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.916400 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.916384 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.916803 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.916787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:23.916921 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.916902 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 05 17:19:23.916983 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.916927 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.917163 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.917143 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s5slm\"" Aug 05 17:19:23.917268 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.917188 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 05 17:19:23.917268 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.917216 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 05 17:19:23.920019 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.919758 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.920019 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.919785 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 05 17:19:23.920189 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.920020 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dzc6c\"" Aug 05 17:19:23.920189 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.920031 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.921933 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.920388 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:23.921933 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.920573 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:23.923671 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923646 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-conf-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.923768 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-multus-certs\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.923768 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-netns\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.923768 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923725 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:23.923925 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.923789 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:23.923925 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-hostroot\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.923925 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923833 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838305c7-2d97-492f-bf25-e2f8f485ae5d-host\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:23.923925 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923850 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-k8s-cni-cncf-io\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.923925 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923901 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-cni-bin\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923940 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-system-cni-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.923980 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-etc-kubernetes\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924042 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tq9p\" (UniqueName: \"kubernetes.io/projected/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-kube-api-access-7tq9p\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924074 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm6fm\" (UniqueName: \"kubernetes.io/projected/cd322441-fd87-4da5-babe-874dda982d83-kube-api-access-mm6fm\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-cni-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924179 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-cnibin\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924204 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-kubelet\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924218 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924220 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-daemon-config\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924239 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838305c7-2d97-492f-bf25-e2f8f485ae5d-serviceca\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924271 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fld4q\" (UniqueName: \"kubernetes.io/projected/838305c7-2d97-492f-bf25-e2f8f485ae5d-kube-api-access-fld4q\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924300 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-os-release\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-cni-binary-copy\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924352 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-socket-dir-parent\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.924612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.924373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-cni-multus\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:23.925593 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.925574 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 17:19:23.927611 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.926961 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:23.927685 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.927649 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 05 17:19:23.929303 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.929263 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 05 17:19:23.929460 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.929313 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c9vsw\"" Aug 05 17:19:23.929460 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.929320 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.929580 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.929540 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:23.929670 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.929655 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.931484 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.931464 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 05 17:19:23.931579 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.931510 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 05 17:19:23.931579 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.931556 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" Aug 05 17:19:23.931748 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.931732 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xpz8w\"" Aug 05 17:19:23.931939 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.931925 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 05 17:19:23.932034 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.932018 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.932068 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.932023 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.932118 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.932104 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 05 17:19:23.932326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.932312 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 05 17:19:23.932744 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.932728 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:23.934754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.934738 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.934976 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.934964 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 05 17:19:23.935140 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.935116 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x2jbd\"" Aug 05 17:19:23.935183 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.935169 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.935744 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.935725 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:23.938089 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.938063 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.938198 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.938138 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.938198 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.938180 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d9zh9\"" Aug 05 17:19:23.938754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.938737 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:23.941309 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.941294 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:23.943011 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.942987 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 05 17:19:23.943107 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.943067 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 05 17:19:23.943107 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.943079 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zvpwb\"" Aug 05 17:19:23.943292 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.943211 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 05 17:19:23.943379 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.943364 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 05 17:19:23.943718 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.943702 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 05 17:19:23.943978 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.943954 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:23.944088 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:23.944015 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:23.944088 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.944026 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-584xh\"" Aug 05 17:19:23.946268 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.946248 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal"] Aug 05 17:19:23.946349 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.946279 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal"] Aug 05 17:19:23.946349 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.946334 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:23.948522 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.948508 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5dt47\"" Aug 05 17:19:23.948681 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.948664 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 05 17:19:23.948735 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.948696 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 05 17:19:23.957901 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.957852 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 17:14:23 +0000 UTC" deadline="2028-05-12 11:38:16.622664826 +0000 UTC" Aug 05 17:19:23.957901 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.957894 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15498h18m52.664773324s" Aug 05 17:19:23.963283 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.963267 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n6ck6" Aug 05 17:19:23.975523 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:23.975477 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n6ck6" Aug 05 17:19:24.018934 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.018915 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 05 17:19:24.024567 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-env-overrides\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.024678 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024575 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-host\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.024678 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024594 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-tuned\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.024678 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.024678 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024640 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysconfig\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5bj\" (UniqueName: \"kubernetes.io/projected/541117ff-f6ae-4c3b-9e99-fe9291664540-kube-api-access-8x5bj\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-cni-bin\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024738 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tq9p\" (UniqueName: \"kubernetes.io/projected/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-kube-api-access-7tq9p\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024759 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-cnibin\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838305c7-2d97-492f-bf25-e2f8f485ae5d-serviceca\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fld4q\" (UniqueName: \"kubernetes.io/projected/838305c7-2d97-492f-bf25-e2f8f485ae5d-kube-api-access-fld4q\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024834 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-var-lib-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024852 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.024876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024874 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-lib-modules\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024901 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-cni-multus\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-cni-binary-copy\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024974 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-cni-multus\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.024981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-netns\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025023 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-netns\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-hostroot\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025079 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzcn2\" (UniqueName: \"kubernetes.io/projected/94edb85c-3031-405a-bf80-76c2d0e4ed88-kube-api-access-fzcn2\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025123 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ncnc\" (UniqueName: \"kubernetes.io/projected/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-kube-api-access-2ncnc\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025119 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-hostroot\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025154 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-run\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-etc-kubernetes\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025201 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovnkube-script-lib\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-socket-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025258 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-etc-kubernetes\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.025457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025287 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-systemd\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025292 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838305c7-2d97-492f-bf25-e2f8f485ae5d-serviceca\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025340 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-kubelet\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025368 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/073ef546-1542-4314-9b81-8fcc9658e539-iptables-alerter-script\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025400 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94edb85c-3031-405a-bf80-76c2d0e4ed88-tmp-dir\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025419 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-kubelet\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025423 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-system-cni-dir\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025466 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-run-netns\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025492 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-node-log\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025518 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-device-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysctl-d\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-os-release\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-multus-certs\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025621 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-etc-selinux\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysctl-conf\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025675 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-sys\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025676 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-multus-certs\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026184 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025687 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-os-release\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025699 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-var-lib-kubelet\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/541117ff-f6ae-4c3b-9e99-fe9291664540-tmp\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838305c7-2d97-492f-bf25-e2f8f485ae5d-host\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025844 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-systemd-units\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025846 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838305c7-2d97-492f-bf25-e2f8f485ae5d-host\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025869 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-log-socket\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025916 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22bf72ca-6129-4438-b017-59817d913812-dbus\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-k8s-cni-cncf-io\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025967 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvjhp\" (UniqueName: \"kubernetes.io/projected/073ef546-1542-4314-9b81-8fcc9658e539-kube-api-access-zvjhp\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.025997 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbpmv\" (UniqueName: \"kubernetes.io/projected/541ddbda-527f-4e49-8236-61e81af9f815-kube-api-access-dbpmv\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026007 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-run-k8s-cni-cncf-io\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-etc-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026044 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovn-node-metrics-cert\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d41c9d97-6ed4-4c96-9587-f20c375cd918-agent-certs\") pod \"konnectivity-agent-ffknk\" (UID: \"d41c9d97-6ed4-4c96-9587-f20c375cd918\") " pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026122 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm6fm\" (UniqueName: \"kubernetes.io/projected/cd322441-fd87-4da5-babe-874dda982d83-kube-api-access-mm6fm\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:24.026880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026157 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-cni-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-daemon-config\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026214 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-os-release\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026237 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-cni-netd\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026262 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026286 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjq7w\" (UniqueName: \"kubernetes.io/projected/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-kube-api-access-qjq7w\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026309 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-kubelet\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026338 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-registration-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/073ef546-1542-4314-9b81-8fcc9658e539-host-slash\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94edb85c-3031-405a-bf80-76c2d0e4ed88-hosts-file\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026396 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-cni-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026444 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-slash\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026480 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-ovn\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026527 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d41c9d97-6ed4-4c96-9587-f20c375cd918-konnectivity-ca\") pod \"konnectivity-agent-ffknk\" (UID: \"d41c9d97-6ed4-4c96-9587-f20c375cd918\") " pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-cni-bin\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026580 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-host-var-lib-cni-bin\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.027555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-system-cni-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026700 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-system-cni-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026713 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-daemon-config\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026747 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-systemd\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-modprobe-d\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026856 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-cnibin\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026898 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovnkube-config\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026935 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-cnibin\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.026972 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-sys-fs\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027000 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-kubernetes\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027025 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22bf72ca-6129-4438-b017-59817d913812-kubelet-config\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027097 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-cni-binary-copy\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027185 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-socket-dir-parent\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.027219 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027259 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-conf-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027309 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-socket-dir-parent\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028705 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.027398 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:24.527360283 +0000 UTC m=+1.928589098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:24.028705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027407 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-multus-conf-dir\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.028705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.027620 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-cni-binary-copy\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.037864 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.037839 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 05 17:19:24.041090 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.041065 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fld4q\" (UniqueName: \"kubernetes.io/projected/838305c7-2d97-492f-bf25-e2f8f485ae5d-kube-api-access-fld4q\") pod \"node-ca-9cf2h\" (UID: \"838305c7-2d97-492f-bf25-e2f8f485ae5d\") " pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.041090 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.041088 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tq9p\" (UniqueName: \"kubernetes.io/projected/2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3-kube-api-access-7tq9p\") pod \"multus-9799r\" (UID: \"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3\") " pod="openshift-multus/multus-9799r" Aug 05 17:19:24.041225 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.041087 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm6fm\" (UniqueName: \"kubernetes.io/projected/cd322441-fd87-4da5-babe-874dda982d83-kube-api-access-mm6fm\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:24.080049 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.080019 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025d669be6d578eb1f9fa3eb786bf236.slice/crio-c865ff523971a9671ec33a4d9319f670735179650367df6d5aeafcaa8f78dba1 WatchSource:0}: Error finding container c865ff523971a9671ec33a4d9319f670735179650367df6d5aeafcaa8f78dba1: Status 404 returned error can't find the container with id c865ff523971a9671ec33a4d9319f670735179650367df6d5aeafcaa8f78dba1 Aug 05 17:19:24.082605 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.082587 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f3b5ee5e09766822cedb87a0da628d.slice/crio-6edc7e3778e4ddd8ff9a20a93a15a1c35902fdd453d25491ef9e0950f264e920 WatchSource:0}: Error finding container 6edc7e3778e4ddd8ff9a20a93a15a1c35902fdd453d25491ef9e0950f264e920: Status 404 returned error can't find the container with id 6edc7e3778e4ddd8ff9a20a93a15a1c35902fdd453d25491ef9e0950f264e920 Aug 05 17:19:24.085066 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.085047 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 17:19:24.127453 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/073ef546-1542-4314-9b81-8fcc9658e539-iptables-alerter-script\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.127453 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127455 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94edb85c-3031-405a-bf80-76c2d0e4ed88-tmp-dir\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127474 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-system-cni-dir\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127499 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-run-netns\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-node-log\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127566 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-device-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127579 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-system-cni-dir\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127593 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysctl-d\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127618 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-etc-selinux\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127641 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysctl-conf\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127656 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-device-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.127679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-sys\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-var-lib-kubelet\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127709 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/541117ff-f6ae-4c3b-9e99-fe9291664540-tmp\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127661 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-run-netns\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127743 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-etc-selinux\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127756 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysctl-d\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127767 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-var-lib-kubelet\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127797 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysctl-conf\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-systemd-units\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-log-socket\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127709 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-sys\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127922 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22bf72ca-6129-4438-b017-59817d913812-dbus\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127953 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvjhp\" (UniqueName: \"kubernetes.io/projected/073ef546-1542-4314-9b81-8fcc9658e539-kube-api-access-zvjhp\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127982 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbpmv\" (UniqueName: \"kubernetes.io/projected/541ddbda-527f-4e49-8236-61e81af9f815-kube-api-access-dbpmv\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.127994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-log-socket\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128007 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22bf72ca-6129-4438-b017-59817d913812-dbus\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.128228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128018 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-etc-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128053 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovn-node-metrics-cert\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128059 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-etc-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128153 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-node-log\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128183 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94edb85c-3031-405a-bf80-76c2d0e4ed88-tmp-dir\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d41c9d97-6ed4-4c96-9587-f20c375cd918-agent-certs\") pod \"konnectivity-agent-ffknk\" (UID: \"d41c9d97-6ed4-4c96-9587-f20c375cd918\") " pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-os-release\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128289 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-cni-netd\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjq7w\" (UniqueName: \"kubernetes.io/projected/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-kube-api-access-qjq7w\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128369 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-kubelet\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128390 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-registration-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128389 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-cni-netd\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128390 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-systemd-units\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128430 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-kubelet\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/073ef546-1542-4314-9b81-8fcc9658e539-host-slash\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128471 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-registration-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.129095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94edb85c-3031-405a-bf80-76c2d0e4ed88-hosts-file\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128503 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128515 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/073ef546-1542-4314-9b81-8fcc9658e539-host-slash\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-os-release\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-slash\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128563 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94edb85c-3031-405a-bf80-76c2d0e4ed88-hosts-file\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128587 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-ovn\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128608 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-slash\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128613 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d41c9d97-6ed4-4c96-9587-f20c375cd918-konnectivity-ca\") pod \"konnectivity-agent-ffknk\" (UID: \"d41c9d97-6ed4-4c96-9587-f20c375cd918\") " pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128645 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128680 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-systemd\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128709 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-modprobe-d\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128688 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-ovn\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128734 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovnkube-config\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-sys-fs\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128806 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-kubernetes\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.129938 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128851 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22bf72ca-6129-4438-b017-59817d913812-kubelet-config\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128919 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-env-overrides\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.128956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-host\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129168 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129223 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/073ef546-1542-4314-9b81-8fcc9658e539-iptables-alerter-script\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129228 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129286 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22bf72ca-6129-4438-b017-59817d913812-kubelet-config\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129288 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovnkube-config\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129331 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-systemd\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129349 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-sys-fs\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129390 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-kubernetes\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129434 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-host\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129436 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-modprobe-d\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-tuned\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysconfig\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129578 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5bj\" (UniqueName: \"kubernetes.io/projected/541117ff-f6ae-4c3b-9e99-fe9291664540-kube-api-access-8x5bj\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.130762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-cni-bin\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129621 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-env-overrides\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129632 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-cnibin\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129661 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129689 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-sysconfig\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129691 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-var-lib-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129729 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129752 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d41c9d97-6ed4-4c96-9587-f20c375cd918-konnectivity-ca\") pod \"konnectivity-agent-ffknk\" (UID: \"d41c9d97-6ed4-4c96-9587-f20c375cd918\") " pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-lib-modules\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129776 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/541ddbda-527f-4e49-8236-61e81af9f815-cnibin\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129793 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-cni-binary-copy\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-lib-modules\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129897 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129820 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-cni-bin\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.129920 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129730 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-var-lib-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129963 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-run-openvswitch\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.129984 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:24.629957662 +0000 UTC m=+2.031186472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130000 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-host-run-ovn-kubernetes\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.129848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzcn2\" (UniqueName: \"kubernetes.io/projected/94edb85c-3031-405a-bf80-76c2d0e4ed88-kube-api-access-fzcn2\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ncnc\" (UniqueName: \"kubernetes.io/projected/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-kube-api-access-2ncnc\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130098 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-run\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovnkube-script-lib\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130180 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-socket-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130203 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130244 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-run\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130314 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/541ddbda-527f-4e49-8236-61e81af9f815-cni-binary-copy\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130369 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-systemd\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130449 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-systemd\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.130520 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-socket-dir\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.131088 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/541117ff-f6ae-4c3b-9e99-fe9291664540-tmp\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.131989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.131162 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovn-node-metrics-cert\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.132504 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.131270 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d41c9d97-6ed4-4c96-9587-f20c375cd918-agent-certs\") pod \"konnectivity-agent-ffknk\" (UID: \"d41c9d97-6ed4-4c96-9587-f20c375cd918\") " pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.132504 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.131441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-ovnkube-script-lib\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.132504 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.132150 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/541117ff-f6ae-4c3b-9e99-fe9291664540-etc-tuned\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.141974 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.141952 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:24.141974 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.141973 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:24.142118 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.141985 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:24.142118 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.142047 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:24.642032025 +0000 UTC m=+2.043260832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:24.143873 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.143853 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbpmv\" (UniqueName: \"kubernetes.io/projected/541ddbda-527f-4e49-8236-61e81af9f815-kube-api-access-dbpmv\") pod \"multus-additional-cni-plugins-9mv62\" (UID: \"541ddbda-527f-4e49-8236-61e81af9f815\") " pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.146523 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.146500 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvjhp\" (UniqueName: \"kubernetes.io/projected/073ef546-1542-4314-9b81-8fcc9658e539-kube-api-access-zvjhp\") pod \"iptables-alerter-49vv7\" (UID: \"073ef546-1542-4314-9b81-8fcc9658e539\") " pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.146849 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.146831 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5bj\" (UniqueName: \"kubernetes.io/projected/541117ff-f6ae-4c3b-9e99-fe9291664540-kube-api-access-8x5bj\") pod \"tuned-kplbj\" (UID: \"541117ff-f6ae-4c3b-9e99-fe9291664540\") " pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.147269 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.147249 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzcn2\" (UniqueName: \"kubernetes.io/projected/94edb85c-3031-405a-bf80-76c2d0e4ed88-kube-api-access-fzcn2\") pod \"node-resolver-9gz9l\" (UID: \"94edb85c-3031-405a-bf80-76c2d0e4ed88\") " pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.147502 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.147486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ncnc\" (UniqueName: \"kubernetes.io/projected/0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61-kube-api-access-2ncnc\") pod \"aws-ebs-csi-driver-node-7qjnf\" (UID: \"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.147800 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.147786 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjq7w\" (UniqueName: \"kubernetes.io/projected/5b8acaa6-4e18-4d02-8f03-d7c62d79b618-kube-api-access-qjq7w\") pod \"ovnkube-node-d2f2g\" (UID: \"5b8acaa6-4e18-4d02-8f03-d7c62d79b618\") " pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.248923 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.248866 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9799r" Aug 05 17:19:24.254203 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.254185 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9cf2h" Aug 05 17:19:24.259182 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.259156 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c72aae8_03ad_49f4_9b8b_ad3c0f87d9f3.slice/crio-a9e5d98aaffe10d37e505367783f9e44c3785dbae9f8414fd4b1c37915b4cd84 WatchSource:0}: Error finding container a9e5d98aaffe10d37e505367783f9e44c3785dbae9f8414fd4b1c37915b4cd84: Status 404 returned error can't find the container with id a9e5d98aaffe10d37e505367783f9e44c3785dbae9f8414fd4b1c37915b4cd84 Aug 05 17:19:24.263868 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.263846 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838305c7_2d97_492f_bf25_e2f8f485ae5d.slice/crio-20a0205a4690eb39c55998642cbb104a493cc268859f0e5c9f80945800b2ca19 WatchSource:0}: Error finding container 20a0205a4690eb39c55998642cbb104a493cc268859f0e5c9f80945800b2ca19: Status 404 returned error can't find the container with id 20a0205a4690eb39c55998642cbb104a493cc268859f0e5c9f80945800b2ca19 Aug 05 17:19:24.264760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.264741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49vv7" Aug 05 17:19:24.268013 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.267998 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:24.275515 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.275482 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073ef546_1542_4314_9b81_8fcc9658e539.slice/crio-d7a2eb8789628d25172f815da05e26cc5d3ad69ed6fba7727ff54c2e6dc13a30 WatchSource:0}: Error finding container d7a2eb8789628d25172f815da05e26cc5d3ad69ed6fba7727ff54c2e6dc13a30: Status 404 returned error can't find the container with id d7a2eb8789628d25172f815da05e26cc5d3ad69ed6fba7727ff54c2e6dc13a30 Aug 05 17:19:24.278460 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.278437 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8acaa6_4e18_4d02_8f03_d7c62d79b618.slice/crio-1956b3f81aacb79695a8f38387f8f8d4fcf24e813e61553f827bc1e7e38e8b9d WatchSource:0}: Error finding container 1956b3f81aacb79695a8f38387f8f8d4fcf24e813e61553f827bc1e7e38e8b9d: Status 404 returned error can't find the container with id 1956b3f81aacb79695a8f38387f8f8d4fcf24e813e61553f827bc1e7e38e8b9d Aug 05 17:19:24.293610 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.293592 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" Aug 05 17:19:24.302771 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.302750 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0035e9eb_c9ea_4a8d_b44f_9cd14cef4a61.slice/crio-84bb2be1c7fa6cbf177f08ff28ebf598c18bf60021c6ec17275f20d66ba58204 WatchSource:0}: Error finding container 84bb2be1c7fa6cbf177f08ff28ebf598c18bf60021c6ec17275f20d66ba58204: Status 404 returned error can't find the container with id 84bb2be1c7fa6cbf177f08ff28ebf598c18bf60021c6ec17275f20d66ba58204 Aug 05 17:19:24.306790 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.306773 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kplbj" Aug 05 17:19:24.314299 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.314279 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541117ff_f6ae_4c3b_9e99_fe9291664540.slice/crio-6c0fbbed5489003315396b2fff2ebe794c1c68f0c9526d80b18747f9a32cdfe4 WatchSource:0}: Error finding container 6c0fbbed5489003315396b2fff2ebe794c1c68f0c9526d80b18747f9a32cdfe4: Status 404 returned error can't find the container with id 6c0fbbed5489003315396b2fff2ebe794c1c68f0c9526d80b18747f9a32cdfe4 Aug 05 17:19:24.324383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.324363 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9gz9l" Aug 05 17:19:24.332302 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.332281 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94edb85c_3031_405a_bf80_76c2d0e4ed88.slice/crio-788916ed37df32c023e0f0ee723298e9ef1ac9e1ff94f75b9dd507cd2279f093 WatchSource:0}: Error finding container 788916ed37df32c023e0f0ee723298e9ef1ac9e1ff94f75b9dd507cd2279f093: Status 404 returned error can't find the container with id 788916ed37df32c023e0f0ee723298e9ef1ac9e1ff94f75b9dd507cd2279f093 Aug 05 17:19:24.348163 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.348142 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9mv62" Aug 05 17:19:24.354366 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.354348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:24.357362 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.357339 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541ddbda_527f_4e49_8236_61e81af9f815.slice/crio-f2922def3e527e8a58149be6acd743c1bb580e9a3dcf23a4588d77cd3a4ac652 WatchSource:0}: Error finding container f2922def3e527e8a58149be6acd743c1bb580e9a3dcf23a4588d77cd3a4ac652: Status 404 returned error can't find the container with id f2922def3e527e8a58149be6acd743c1bb580e9a3dcf23a4588d77cd3a4ac652 Aug 05 17:19:24.362380 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:24.362359 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41c9d97_6ed4_4c96_9587_f20c375cd918.slice/crio-8575091ff2a45a8fd1c656da01d15ab023d7e38a0e3cad65055ba6d5da224abb WatchSource:0}: Error finding container 8575091ff2a45a8fd1c656da01d15ab023d7e38a0e3cad65055ba6d5da224abb: Status 404 returned error can't find the container with id 8575091ff2a45a8fd1c656da01d15ab023d7e38a0e3cad65055ba6d5da224abb Aug 05 17:19:24.534044 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.533962 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:24.534210 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.534181 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:24.534281 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.534241 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:25.534222153 +0000 UTC m=+2.935450953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:24.634468 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.634435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:24.634653 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.634572 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:24.634653 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.634635 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:25.634614825 +0000 UTC m=+3.035843619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:24.738111 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.738004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:24.738331 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.738196 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:24.738331 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.738222 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:24.738331 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.738237 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:24.738331 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:24.738301 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:25.738279966 +0000 UTC m=+3.139508763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:24.979735 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.979599 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:14:24 +0000 UTC" deadline="2028-02-17 16:45:38.467016918 +0000 UTC" Aug 05 17:19:24.979735 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:24.979639 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13463h26m13.487382515s" Aug 05 17:19:25.021812 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.021268 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:25.021812 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.021418 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:25.022510 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.022212 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:25.022510 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.022322 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:25.047907 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.047846 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffknk" event={"ID":"d41c9d97-6ed4-4c96-9587-f20c375cd918","Type":"ContainerStarted","Data":"8575091ff2a45a8fd1c656da01d15ab023d7e38a0e3cad65055ba6d5da224abb"} Aug 05 17:19:25.056021 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.055952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9gz9l" event={"ID":"94edb85c-3031-405a-bf80-76c2d0e4ed88","Type":"ContainerStarted","Data":"788916ed37df32c023e0f0ee723298e9ef1ac9e1ff94f75b9dd507cd2279f093"} Aug 05 17:19:25.067202 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.067115 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kplbj" event={"ID":"541117ff-f6ae-4c3b-9e99-fe9291664540","Type":"ContainerStarted","Data":"6c0fbbed5489003315396b2fff2ebe794c1c68f0c9526d80b18747f9a32cdfe4"} Aug 05 17:19:25.090557 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.090473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" event={"ID":"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61","Type":"ContainerStarted","Data":"84bb2be1c7fa6cbf177f08ff28ebf598c18bf60021c6ec17275f20d66ba58204"} Aug 05 17:19:25.097140 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.097095 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49vv7" event={"ID":"073ef546-1542-4314-9b81-8fcc9658e539","Type":"ContainerStarted","Data":"d7a2eb8789628d25172f815da05e26cc5d3ad69ed6fba7727ff54c2e6dc13a30"} Aug 05 17:19:25.107889 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.107809 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9cf2h" event={"ID":"838305c7-2d97-492f-bf25-e2f8f485ae5d","Type":"ContainerStarted","Data":"20a0205a4690eb39c55998642cbb104a493cc268859f0e5c9f80945800b2ca19"} Aug 05 17:19:25.118641 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.118587 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9799r" event={"ID":"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3","Type":"ContainerStarted","Data":"a9e5d98aaffe10d37e505367783f9e44c3785dbae9f8414fd4b1c37915b4cd84"} Aug 05 17:19:25.127546 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.127513 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" event={"ID":"025d669be6d578eb1f9fa3eb786bf236","Type":"ContainerStarted","Data":"c865ff523971a9671ec33a4d9319f670735179650367df6d5aeafcaa8f78dba1"} Aug 05 17:19:25.130324 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.130213 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerStarted","Data":"f2922def3e527e8a58149be6acd743c1bb580e9a3dcf23a4588d77cd3a4ac652"} Aug 05 17:19:25.136195 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.135915 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"1956b3f81aacb79695a8f38387f8f8d4fcf24e813e61553f827bc1e7e38e8b9d"} Aug 05 17:19:25.144358 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.144333 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" event={"ID":"47f3b5ee5e09766822cedb87a0da628d","Type":"ContainerStarted","Data":"6edc7e3778e4ddd8ff9a20a93a15a1c35902fdd453d25491ef9e0950f264e920"} Aug 05 17:19:25.152632 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.152427 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 17:19:25.544284 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.544205 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:25.544488 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.544412 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:25.544488 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.544477 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:27.544457123 +0000 UTC m=+4.945685921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:25.644795 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.644754 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:25.644980 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.644932 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:25.645039 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.644995 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:27.644976385 +0000 UTC m=+5.046205182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:25.745440 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.745403 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:25.745632 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.745619 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:25.745692 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.745643 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:25.745692 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.745656 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:25.745790 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:25.745716 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:27.745698125 +0000 UTC m=+5.146926934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:25.980408 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.980315 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:14:24 +0000 UTC" deadline="2028-04-19 03:47:22.665098628 +0000 UTC" Aug 05 17:19:25.980408 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:25.980358 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14938h27m56.684745473s" Aug 05 17:19:26.019896 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:26.019858 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:26.020080 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:26.020003 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:26.410095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:26.409739 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 05 17:19:27.020399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:27.020303 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:27.020871 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.020449 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:27.020942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:27.020897 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:27.022488 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.021001 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:27.559566 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:27.559526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:27.559758 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.559679 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:27.559758 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.559756 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:31.559736311 +0000 UTC m=+8.960965108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:27.661326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:27.660678 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:27.661326 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.660861 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:27.661326 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.660920 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:31.660902644 +0000 UTC m=+9.062131441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:27.762111 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:27.762058 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:27.762314 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.762241 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:27.762314 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.762262 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:27.762314 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.762274 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:27.762477 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:27.762329 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:31.762308625 +0000 UTC m=+9.163537434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:28.020695 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:28.020614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:28.021217 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:28.020749 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:29.021296 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:29.021259 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:29.021755 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:29.021411 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:29.023411 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:29.023271 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:29.023411 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:29.023377 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:30.020471 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:30.020341 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:30.020666 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:30.020488 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:31.020358 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:31.020321 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:31.022047 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.022008 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:31.024926 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:31.020380 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:31.024926 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.023029 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:31.595577 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:31.595533 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:31.595767 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.595689 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:31.595767 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.595764 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:39.595744359 +0000 UTC m=+16.996973156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:31.696211 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:31.696047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:31.696211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.696178 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:31.696460 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.696246 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:39.696230709 +0000 UTC m=+17.097459506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:31.797383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:31.797330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:31.797579 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.797497 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:31.797579 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.797523 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:31.797579 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.797538 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:31.797770 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:31.797600 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:39.797580531 +0000 UTC m=+17.198809346 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:32.020854 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:32.020759 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:32.021354 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:32.020906 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:33.020713 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:33.020657 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:33.022055 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:33.021731 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:33.022055 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:33.021848 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:33.022055 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:33.021979 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:34.020263 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:34.020229 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:34.020469 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:34.020372 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:35.020175 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:35.020142 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:35.020541 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:35.020143 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:35.020541 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:35.020281 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:35.020541 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:35.020348 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:36.020436 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:36.020402 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:36.020871 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:36.020524 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:37.020897 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:37.020863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:37.021342 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:37.020870 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:37.021342 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:37.021042 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:37.021342 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:37.021075 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:38.020777 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:38.020745 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:38.020926 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:38.020880 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:39.020286 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:39.020248 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:39.020459 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.020395 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:39.020459 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:39.020402 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:39.020564 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.020497 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:39.662821 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:39.662773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:39.663356 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.662953 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:39.663356 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.663025 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:55.663007243 +0000 UTC m=+33.064236035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:39.763771 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:39.763730 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:39.763959 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.763901 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:39.764003 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.763968 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:55.763953327 +0000 UTC m=+33.165182119 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:39.864558 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:39.864523 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:39.864774 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.864705 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:39.864774 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.864729 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:39.864774 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.864742 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:39.864940 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:39.864804 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:55.864789477 +0000 UTC m=+33.266018274 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:40.020805 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:40.020741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:40.020948 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:40.020836 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:41.020065 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:41.020022 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:41.020510 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:41.020023 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:41.020510 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:41.020156 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:41.020510 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:41.020217 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:42.020155 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:42.020123 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:42.020633 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:42.020242 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:43.021105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:43.021070 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:43.021557 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:43.021112 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:43.021557 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:43.021195 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:43.021557 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:43.021464 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:44.020481 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:44.020446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:44.020633 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:44.020546 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:45.020885 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.020707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:45.021193 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.020704 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:45.021193 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:45.020982 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:45.021193 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:45.021086 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:45.191100 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.190864 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kplbj" event={"ID":"541117ff-f6ae-4c3b-9e99-fe9291664540","Type":"ContainerStarted","Data":"9a56f64cebb56340934a697c278c128db297704205bd3f502ab1e6231ba7ac32"} Aug 05 17:19:45.192368 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.192346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9799r" event={"ID":"2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3","Type":"ContainerStarted","Data":"d88769d4536763d470b7e5ea54d4014c0b414b013719a64e1e5e6d30bc2b28fa"} Aug 05 17:19:45.195574 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.195559 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:19:45.195883 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.195866 2575 generic.go:356] "Generic (PLEG): container finished" podID="5b8acaa6-4e18-4d02-8f03-d7c62d79b618" containerID="9cf84fd7dfd9111611a2a60f4fd12c1a0a15499707fc2ed1d05180f8e3176e13" exitCode=1 Aug 05 17:19:45.195937 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.195921 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"42e309c38c3d88cc9311ae5869756787ac3f7759b8ff08db6bac139ae51428af"} Aug 05 17:19:45.195986 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.195974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"37c869151a17f89a240679cad92b5417492b8b3d1358eff2a06dd7c9f27d064d"} Aug 05 17:19:45.196030 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.196018 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"85adb1132dde0a8903197d72eff27c0b36a3edb43cbe6ee67670994daea34c93"} Aug 05 17:19:45.196083 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.196069 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerDied","Data":"9cf84fd7dfd9111611a2a60f4fd12c1a0a15499707fc2ed1d05180f8e3176e13"} Aug 05 17:19:45.196130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.196111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"0251af901a58171d1a2c6908272dd62756b12e04fbcf846076c369e7c91ba517"} Aug 05 17:19:45.197196 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.197179 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" event={"ID":"47f3b5ee5e09766822cedb87a0da628d","Type":"ContainerStarted","Data":"2985348f1b16986309e22a063bbdb0cd648f8648bd6ef40a0e7706a45fa27e69"} Aug 05 17:19:45.225698 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.225659 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kplbj" podStartSLOduration=1.371216545 podStartE2EDuration="21.22564705s" totalImagesPullingTime="19.854430505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:45.225268669 +0000 UTC m=+22.626497482" watchObservedRunningTime="2026-08-05 17:19:45.22564705 +0000 UTC m=+22.626875863" Aug 05 17:19:45.237948 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.237915 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-58.ec2.internal" podStartSLOduration=21.237900275 podStartE2EDuration="21.237900275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:45.23775548 +0000 UTC m=+22.638984288" watchObservedRunningTime="2026-08-05 17:19:45.237900275 +0000 UTC m=+22.639129088" Aug 05 17:19:45.254807 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:45.254722 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9799r" podStartSLOduration=2.314917083 podStartE2EDuration="22.254708121s" totalImagesPullingTime="19.939791038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:45.254311447 +0000 UTC m=+22.655540262" watchObservedRunningTime="2026-08-05 17:19:45.254708121 +0000 UTC m=+22.655936935" Aug 05 17:19:46.020776 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.020561 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:46.020959 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:46.020856 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:46.202537 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.202508 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:19:46.202960 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.202936 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"48fe95253f0830fef932fbd364123518e78f6fe62997a8aef743736817a07de6"} Aug 05 17:19:46.204501 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.204474 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffknk" event={"ID":"d41c9d97-6ed4-4c96-9587-f20c375cd918","Type":"ContainerStarted","Data":"e6231f3bd5ae075dc66fd7b4a515db98bb7dde15c8759844fcaf1d0b41e0cc7c"} Aug 05 17:19:46.206003 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.205979 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9gz9l" event={"ID":"94edb85c-3031-405a-bf80-76c2d0e4ed88","Type":"ContainerStarted","Data":"12434cc8219c1825b85f4ce171894941c7747d2e63ee79fcd69289642af6ba75"} Aug 05 17:19:46.207578 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.207554 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" event={"ID":"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61","Type":"ContainerStarted","Data":"85e5d529fc5418cc33f3544175a90150ee5834d745a2046539ea7385b1a269a0"} Aug 05 17:19:46.209194 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.209171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49vv7" event={"ID":"073ef546-1542-4314-9b81-8fcc9658e539","Type":"ContainerStarted","Data":"779054926e93818e21d0642759ce56786e34301807b810433de7b3c81659e4f0"} Aug 05 17:19:46.210680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.210658 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9cf2h" event={"ID":"838305c7-2d97-492f-bf25-e2f8f485ae5d","Type":"ContainerStarted","Data":"58df1d91b29ee7cc1fbd5df1ad08f96538cb786c716a1d78f1ca85233a046d81"} Aug 05 17:19:46.212358 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.212332 2575 generic.go:356] "Generic (PLEG): container finished" podID="025d669be6d578eb1f9fa3eb786bf236" containerID="778ed80a126593d3455ecfaa95ef550f4e9604eefb031e109fde69258986ff00" exitCode=0 Aug 05 17:19:46.212450 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.212414 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" event={"ID":"025d669be6d578eb1f9fa3eb786bf236","Type":"ContainerDied","Data":"778ed80a126593d3455ecfaa95ef550f4e9604eefb031e109fde69258986ff00"} Aug 05 17:19:46.214045 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.214027 2575 generic.go:356] "Generic (PLEG): container finished" podID="541ddbda-527f-4e49-8236-61e81af9f815" containerID="4c8c852b072085819726f4beb9c6ce1a2d312a0c8b44c8f9e96e5496a2d1d477" exitCode=0 Aug 05 17:19:46.214154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.214095 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerDied","Data":"4c8c852b072085819726f4beb9c6ce1a2d312a0c8b44c8f9e96e5496a2d1d477"} Aug 05 17:19:46.219758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.219712 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ffknk" podStartSLOduration=2.420225172 podStartE2EDuration="22.219695649s" totalImagesPullingTime="19.799470477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:46.219356485 +0000 UTC m=+23.620585300" watchObservedRunningTime="2026-08-05 17:19:46.219695649 +0000 UTC m=+23.620924466" Aug 05 17:19:46.320975 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.320930 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9cf2h" podStartSLOduration=3.423088383 podStartE2EDuration="23.320915001s" totalImagesPullingTime="19.897826618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:46.290941619 +0000 UTC m=+23.692170434" watchObservedRunningTime="2026-08-05 17:19:46.320915001 +0000 UTC m=+23.722143814" Aug 05 17:19:46.321283 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.321259 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-49vv7" podStartSLOduration=2.435151787 podStartE2EDuration="22.321253133s" totalImagesPullingTime="19.886101346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:46.321123972 +0000 UTC m=+23.722352785" watchObservedRunningTime="2026-08-05 17:19:46.321253133 +0000 UTC m=+23.722481946" Aug 05 17:19:46.663960 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.663933 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 05 17:19:46.775365 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.775331 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:46.775988 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.775962 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:46.792996 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.792951 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9gz9l" podStartSLOduration=2.952163128 podStartE2EDuration="22.792935164s" totalImagesPullingTime="19.840772036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:46.340312741 +0000 UTC m=+23.741541555" watchObservedRunningTime="2026-08-05 17:19:46.792935164 +0000 UTC m=+24.194164009" Aug 05 17:19:46.990774 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.990612 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T17:19:46.663950807Z","UUID":"ba2c8a0c-fe2e-42e7-997a-2cca38066eeb","Handler":null,"Name":"","Endpoint":""} Aug 05 17:19:46.992520 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.992496 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 05 17:19:46.992657 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:46.992542 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 05 17:19:47.020840 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.020812 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:47.021006 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.020855 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:47.021006 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:47.020944 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:47.021485 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:47.021094 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:47.217961 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.217931 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" event={"ID":"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61","Type":"ContainerStarted","Data":"4293d69af7580f89bd3848a417c69fcc0a31c4219f3931b123827309f4e424af"} Aug 05 17:19:47.220100 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.220068 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" event={"ID":"025d669be6d578eb1f9fa3eb786bf236","Type":"ContainerStarted","Data":"fe8d84f6942cc95ff6f894ac6c33d54ab6e7c3f9ab2ef5cf63ab959a17970f1f"} Aug 05 17:19:47.220798 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.220775 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:47.221153 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.221135 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ffknk" Aug 05 17:19:47.258042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:47.257969 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-58.ec2.internal" podStartSLOduration=23.257955874 podStartE2EDuration="23.257955874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:47.257571681 +0000 UTC m=+24.658800495" watchObservedRunningTime="2026-08-05 17:19:47.257955874 +0000 UTC m=+24.659184687" Aug 05 17:19:48.020508 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:48.020476 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:48.020772 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:48.020605 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:48.226188 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:48.226160 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:19:48.226674 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:48.226647 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"750515bfcea4ec662f43edf23590fcf7daf6b667fd449d17d2a24d430bf5de44"} Aug 05 17:19:49.023006 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:49.022975 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:49.023263 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:49.023012 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:49.023263 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:49.023114 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:49.023364 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:49.023240 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:49.231681 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:49.231642 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" event={"ID":"0035e9eb-c9ea-4a8d-b44f-9cd14cef4a61","Type":"ContainerStarted","Data":"7524e01972d476d7182c1d28d81099f4cb4d025c4853c3297f1566b5c4a57307"} Aug 05 17:19:49.251782 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:49.251728 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7qjnf" podStartSLOduration=2.518649934 podStartE2EDuration="25.251707501s" totalImagesPullingTime="22.733057567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:49.251673931 +0000 UTC m=+26.652902745" watchObservedRunningTime="2026-08-05 17:19:49.251707501 +0000 UTC m=+26.652936313" Aug 05 17:19:50.020324 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:50.020194 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:50.020458 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:50.020397 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:50.238650 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:50.238623 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:19:50.239326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:50.239001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"5d78aac197642f0c59e34ba88c56e4bd54c56af9558aaeef8fb90dccf0bf066c"} Aug 05 17:19:50.239561 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:50.239545 2575 scope.go:117] "RemoveContainer" containerID="9cf84fd7dfd9111611a2a60f4fd12c1a0a15499707fc2ed1d05180f8e3176e13" Aug 05 17:19:51.022919 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.022897 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:51.023038 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:51.022981 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:51.023038 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.023006 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:51.023156 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:51.023078 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:51.242335 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.242251 2575 generic.go:356] "Generic (PLEG): container finished" podID="541ddbda-527f-4e49-8236-61e81af9f815" containerID="28a01e1207b4489af5404a0da587edb28a96d01ec220d0d8dccb5848c23f132b" exitCode=0 Aug 05 17:19:51.242927 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.242340 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerDied","Data":"28a01e1207b4489af5404a0da587edb28a96d01ec220d0d8dccb5848c23f132b"} Aug 05 17:19:51.245993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.245977 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:19:51.246349 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.246327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" event={"ID":"5b8acaa6-4e18-4d02-8f03-d7c62d79b618","Type":"ContainerStarted","Data":"f4587cb0c824b179a46f377175e852bfd3c03479564b1f3a770ae12e0da433db"} Aug 05 17:19:51.246440 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.246396 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 05 17:19:51.246543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.246527 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:51.246609 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.246596 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:51.261482 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.261462 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:51.261581 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.261556 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:51.296563 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:51.296511 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" podStartSLOduration=6.944885914 podStartE2EDuration="27.296493369s" totalImagesPullingTime="20.351607455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:19:51.295855987 +0000 UTC m=+28.697084801" watchObservedRunningTime="2026-08-05 17:19:51.296493369 +0000 UTC m=+28.697722184" Aug 05 17:19:52.020247 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.020076 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:52.020436 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:52.020328 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:52.250337 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.250304 2575 generic.go:356] "Generic (PLEG): container finished" podID="541ddbda-527f-4e49-8236-61e81af9f815" containerID="81c75bf162c5c344d17e25a56503868559e2c27239a8c6fe28daba9a70c96783" exitCode=0 Aug 05 17:19:52.250750 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.250352 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerDied","Data":"81c75bf162c5c344d17e25a56503868559e2c27239a8c6fe28daba9a70c96783"} Aug 05 17:19:52.250750 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.250453 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 05 17:19:52.312932 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.312905 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sgrxg"] Aug 05 17:19:52.313113 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.313032 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:52.313187 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:52.313135 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:52.315903 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.315881 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dpb7v"] Aug 05 17:19:52.316022 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.315969 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:52.316097 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:52.316037 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:52.319460 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.319435 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2m4bd"] Aug 05 17:19:52.319555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:52.319529 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:52.319639 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:52.319618 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:53.252677 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:53.252650 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 05 17:19:54.020083 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:54.020037 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:54.020235 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:54.020088 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:54.020235 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:54.020088 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:54.020235 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:54.020179 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:54.020362 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:54.020247 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:54.020362 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:54.020320 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:54.066302 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:54.066269 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:19:54.257441 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:54.257412 2575 generic.go:356] "Generic (PLEG): container finished" podID="541ddbda-527f-4e49-8236-61e81af9f815" containerID="7476857a320e2d8826cab65fa99bb5ad1d0c9a2ab761ead6ce583cd8fa47ded7" exitCode=0 Aug 05 17:19:54.257834 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:54.257495 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerDied","Data":"7476857a320e2d8826cab65fa99bb5ad1d0c9a2ab761ead6ce583cd8fa47ded7"} Aug 05 17:19:55.679422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:55.679388 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:55.679891 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.679511 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:55.679891 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.679571 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.679554902 +0000 UTC m=+65.080783699 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 05 17:19:55.780421 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:55.780386 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:55.780583 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.780544 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:55.780629 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.780608 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret podName:22bf72ca-6129-4438-b017-59817d913812 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.78059347 +0000 UTC m=+65.181822262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret") pod "global-pull-secret-syncer-dpb7v" (UID: "22bf72ca-6129-4438-b017-59817d913812") : object "kube-system"/"original-pull-secret" not registered Aug 05 17:19:55.881323 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:55.881290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:55.881503 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.881445 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 05 17:19:55.881503 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.881466 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 05 17:19:55.881503 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.881483 2575 projected.go:196] Error preparing data for projected volume kube-api-access-gs7jp for pod openshift-network-diagnostics/network-check-target-sgrxg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:55.881629 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:55.881542 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp podName:99e75228-e8bf-44e8-9781-1121221d15b1 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.881524441 +0000 UTC m=+65.282753455 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gs7jp" (UniqueName: "kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp") pod "network-check-target-sgrxg" (UID: "99e75228-e8bf-44e8-9781-1121221d15b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 05 17:19:56.020729 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:56.020704 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:56.020871 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:56.020704 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:56.020871 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:56.020819 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:56.020988 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:56.020708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:56.020988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:56.020899 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:56.020988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:56.020973 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:58.020892 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.020718 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:19:58.021304 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.020739 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:19:58.021304 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.020979 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgrxg" podUID="99e75228-e8bf-44e8-9781-1121221d15b1" Aug 05 17:19:58.021304 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.020739 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:19:58.021304 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.021091 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dpb7v" podUID="22bf72ca-6129-4438-b017-59817d913812" Aug 05 17:19:58.021304 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.021171 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2m4bd" podUID="cd322441-fd87-4da5-babe-874dda982d83" Aug 05 17:19:58.270711 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.270677 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerStarted","Data":"ffa4c9da5d9efcf2183d20f9d667e9a630e4d0becc5d85dafae72fdb3d24fdfd"} Aug 05 17:19:58.387538 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.387514 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-58.ec2.internal" event="NodeReady" Aug 05 17:19:58.387682 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.387671 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 05 17:19:58.425624 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.425564 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-765d5c5557-4zdr8"] Aug 05 17:19:58.428317 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.428303 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.434094 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.434067 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 05 17:19:58.434197 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.434079 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 05 17:19:58.434197 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.434146 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 05 17:19:58.434419 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.434337 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nj4rs\"" Aug 05 17:19:58.439743 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.439725 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 05 17:19:58.448189 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.448169 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-765d5c5557-4zdr8"] Aug 05 17:19:58.450485 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.450466 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cb75m"] Aug 05 17:19:58.453349 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.453329 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p8tz8"] Aug 05 17:19:58.453457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.453442 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.455711 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.455693 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 05 17:19:58.455784 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.455716 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 05 17:19:58.455784 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.455722 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6sjj\"" Aug 05 17:19:58.455897 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.455882 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.457966 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.457950 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 05 17:19:58.458254 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.458233 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 05 17:19:58.458349 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.458258 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 05 17:19:58.458513 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.458498 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rskhh\"" Aug 05 17:19:58.465844 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.465445 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p8tz8"] Aug 05 17:19:58.469503 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.468181 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cb75m"] Aug 05 17:19:58.603872 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.603840 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.604018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.603880 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-trusted-ca\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.603902 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-tmp-dir\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.604018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.603974 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72739ecf-e267-4cc8-b2f3-be3d56767b3b-ca-trust-extracted\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-bound-sa-token\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604027 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-config-volume\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.604168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604044 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjpj\" (UniqueName: \"kubernetes.io/projected/298eeb1d-3ed1-407f-990f-08b90ccd84f5-kube-api-access-fkjpj\") pod \"ingress-canary-p8tz8\" (UID: \"298eeb1d-3ed1-407f-990f-08b90ccd84f5\") " pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.604168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-installation-pull-secrets\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604143 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-image-registry-private-configuration\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-certificates\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.604281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604215 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/298eeb1d-3ed1-407f-990f-08b90ccd84f5-cert\") pod \"ingress-canary-p8tz8\" (UID: \"298eeb1d-3ed1-407f-990f-08b90ccd84f5\") " pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.604281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8rsx\" (UniqueName: \"kubernetes.io/projected/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-kube-api-access-z8rsx\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.604281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.604267 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-866w4\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-kube-api-access-866w4\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705259 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705180 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-trusted-ca\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705259 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-tmp-dir\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.705459 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72739ecf-e267-4cc8-b2f3-be3d56767b3b-ca-trust-extracted\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705459 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705295 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-bound-sa-token\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705459 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-config-volume\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.705459 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705350 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjpj\" (UniqueName: \"kubernetes.io/projected/298eeb1d-3ed1-407f-990f-08b90ccd84f5-kube-api-access-fkjpj\") pod \"ingress-canary-p8tz8\" (UID: \"298eeb1d-3ed1-407f-990f-08b90ccd84f5\") " pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.705459 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705386 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-installation-pull-secrets\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-image-registry-private-configuration\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-certificates\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705597 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/298eeb1d-3ed1-407f-990f-08b90ccd84f5-cert\") pod \"ingress-canary-p8tz8\" (UID: \"298eeb1d-3ed1-407f-990f-08b90ccd84f5\") " pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.705680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8rsx\" (UniqueName: \"kubernetes.io/projected/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-kube-api-access-z8rsx\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.705680 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705664 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-866w4\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-kube-api-access-866w4\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.705744 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.705761 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705761 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72739ecf-e267-4cc8-b2f3-be3d56767b3b-ca-trust-extracted\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.705796 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.705812 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:19:59.205794851 +0000 UTC m=+36.607023662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:58.705855 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:19:59.205838702 +0000 UTC m=+36.607067515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705661 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-tmp-dir\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.705952 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.705935 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-config-volume\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.706386 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.706253 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-trusted-ca\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.706386 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.706326 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-certificates\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.709744 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.709720 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-installation-pull-secrets\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.709853 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.709762 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-image-registry-private-configuration\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.709853 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.709826 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/298eeb1d-3ed1-407f-990f-08b90ccd84f5-cert\") pod \"ingress-canary-p8tz8\" (UID: \"298eeb1d-3ed1-407f-990f-08b90ccd84f5\") " pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.714403 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.714379 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-866w4\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-kube-api-access-866w4\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.715514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.715493 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjpj\" (UniqueName: \"kubernetes.io/projected/298eeb1d-3ed1-407f-990f-08b90ccd84f5-kube-api-access-fkjpj\") pod \"ingress-canary-p8tz8\" (UID: \"298eeb1d-3ed1-407f-990f-08b90ccd84f5\") " pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.716119 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.716098 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8rsx\" (UniqueName: \"kubernetes.io/projected/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-kube-api-access-z8rsx\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:58.716802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.716784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-bound-sa-token\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:58.771844 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.771815 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p8tz8" Aug 05 17:19:58.900660 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:58.900628 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p8tz8"] Aug 05 17:19:58.904784 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:19:58.904755 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298eeb1d_3ed1_407f_990f_08b90ccd84f5.slice/crio-3a8e612f9f36614bd2c08de7195b82afec53a2679832dc73d89f1a27ebae35a9 WatchSource:0}: Error finding container 3a8e612f9f36614bd2c08de7195b82afec53a2679832dc73d89f1a27ebae35a9: Status 404 returned error can't find the container with id 3a8e612f9f36614bd2c08de7195b82afec53a2679832dc73d89f1a27ebae35a9 Aug 05 17:19:59.209508 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:59.209473 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:19:59.210311 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:59.209556 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:19:59.210311 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:59.209666 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:19:59.210311 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:59.209682 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:19:59.210311 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:59.209698 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:19:59.210311 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:59.209742 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:20:00.209722306 +0000 UTC m=+37.610951113 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:19:59.210311 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:19:59.209768 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:00.209748806 +0000 UTC m=+37.610977617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:19:59.276612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:59.276578 2575 generic.go:356] "Generic (PLEG): container finished" podID="541ddbda-527f-4e49-8236-61e81af9f815" containerID="ffa4c9da5d9efcf2183d20f9d667e9a630e4d0becc5d85dafae72fdb3d24fdfd" exitCode=0 Aug 05 17:19:59.276778 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:59.276660 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerDied","Data":"ffa4c9da5d9efcf2183d20f9d667e9a630e4d0becc5d85dafae72fdb3d24fdfd"} Aug 05 17:19:59.277941 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:19:59.277918 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p8tz8" event={"ID":"298eeb1d-3ed1-407f-990f-08b90ccd84f5","Type":"ContainerStarted","Data":"3a8e612f9f36614bd2c08de7195b82afec53a2679832dc73d89f1a27ebae35a9"} Aug 05 17:20:00.020590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.020404 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:20:00.020784 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.020413 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:20:00.020784 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.020450 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:20:00.023395 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.023375 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 05 17:20:00.023874 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.023861 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 05 17:20:00.024045 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.023890 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmhgk\"" Aug 05 17:20:00.024045 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.023861 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 05 17:20:00.024045 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.023973 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 05 17:20:00.024212 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.024091 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4rdxm\"" Aug 05 17:20:00.220658 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.220624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:20:00.221083 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.220714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:20:00.221083 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:00.220796 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:20:00.221083 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:00.220818 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:20:00.221083 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:00.220880 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:20:02.220863425 +0000 UTC m=+39.622092217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:20:00.221083 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:00.220882 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:20:00.221357 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:00.221328 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:02.220931333 +0000 UTC m=+39.622160127 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:20:00.284802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.284715 2575 generic.go:356] "Generic (PLEG): container finished" podID="541ddbda-527f-4e49-8236-61e81af9f815" containerID="c6ece4fad22f8e03d865214729072ac1aeabcfa18a1ae7eedd4aa1a6b8f82a30" exitCode=0 Aug 05 17:20:00.284802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:00.284779 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerDied","Data":"c6ece4fad22f8e03d865214729072ac1aeabcfa18a1ae7eedd4aa1a6b8f82a30"} Aug 05 17:20:01.289882 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:01.289846 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9mv62" event={"ID":"541ddbda-527f-4e49-8236-61e81af9f815","Type":"ContainerStarted","Data":"f6ba2ce6c195a13d149e9670c18d6679e505d46353c30d7aff08ef9ba6e23f3e"} Aug 05 17:20:01.291128 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:01.291104 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p8tz8" event={"ID":"298eeb1d-3ed1-407f-990f-08b90ccd84f5","Type":"ContainerStarted","Data":"112c8a1fbad6bac3b1e5b9f693c6f6888ba8136d48a95779b2b51e034de7fe03"} Aug 05 17:20:01.316413 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:01.316371 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9mv62" podStartSLOduration=7.487052999 podStartE2EDuration="37.316356612s" totalImagesPullingTime="29.829303613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:01.315730007 +0000 UTC m=+38.716958819" watchObservedRunningTime="2026-08-05 17:20:01.316356612 +0000 UTC m=+38.717585425" Aug 05 17:20:01.331389 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:01.331349 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p8tz8" podStartSLOduration=1.381627102 podStartE2EDuration="3.331339345s" totalImagesPullingTime="1.949712243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:01.330579613 +0000 UTC m=+38.731808440" watchObservedRunningTime="2026-08-05 17:20:01.331339345 +0000 UTC m=+38.732568158" Aug 05 17:20:02.236785 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:02.236750 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:20:02.236993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:02.236811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:20:02.236993 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:02.236910 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:20:02.236993 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:02.236925 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:20:02.236993 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:02.236946 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:20:02.236993 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:02.236960 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:06.2369473 +0000 UTC m=+43.638176092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:20:02.237211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:02.236999 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:20:06.236983449 +0000 UTC m=+43.638212280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:20:06.265331 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:06.265292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:20:06.265741 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:06.265368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:20:06.265741 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:06.265452 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:20:06.265741 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:06.265518 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:14.265503822 +0000 UTC m=+51.666732615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:20:06.265741 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:06.265455 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:20:06.265741 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:06.265561 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:20:06.265741 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:06.265613 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:20:14.265598894 +0000 UTC m=+51.666827692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:20:14.320641 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:14.320590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:20:14.321211 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:14.320669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:20:14.321211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:14.320739 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:20:14.321211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:14.320760 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:20:14.321211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:14.320769 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:20:14.321211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:14.320805 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.320790501 +0000 UTC m=+67.722019293 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:20:14.321211 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:14.320818 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.320812591 +0000 UTC m=+67.722041383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:20:22.370598 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.370568 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-mxs75"] Aug 05 17:20:22.400753 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.400725 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf"] Aug 05 17:20:22.400920 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.400860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.403423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.403400 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Aug 05 17:20:22.403556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.403441 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Aug 05 17:20:22.403556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.403543 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-n2qn7\"" Aug 05 17:20:22.403652 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.403636 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Aug 05 17:20:22.403703 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.403688 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:20:22.408489 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.408470 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Aug 05 17:20:22.414662 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.414640 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-75ch2"] Aug 05 17:20:22.414786 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.414770 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:22.417189 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.417169 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 05 17:20:22.417280 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.417188 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 05 17:20:22.417412 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.417390 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dkrzv\"" Aug 05 17:20:22.417508 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.417451 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:20:22.428717 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.428700 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf"] Aug 05 17:20:22.428791 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.428775 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.428826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.428803 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-75ch2"] Aug 05 17:20:22.428857 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.428831 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-mxs75"] Aug 05 17:20:22.431382 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.431363 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 05 17:20:22.431464 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.431366 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Aug 05 17:20:22.431780 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.431764 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 05 17:20:22.431780 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.431774 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lgdv6\"" Aug 05 17:20:22.431894 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.431799 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Aug 05 17:20:22.436872 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.436853 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Aug 05 17:20:22.468310 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.468288 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd"] Aug 05 17:20:22.479140 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479116 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/451a1bd3-fbed-42c5-a04e-45a6befe04f1-serving-cert\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.479231 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479146 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/451a1bd3-fbed-42c5-a04e-45a6befe04f1-trusted-ca\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.479231 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479164 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36009ed7-f9b0-48d3-920e-3d6d452d18e8-trusted-ca-bundle\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.479231 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479193 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx22n\" (UniqueName: \"kubernetes.io/projected/451a1bd3-fbed-42c5-a04e-45a6befe04f1-kube-api-access-xx22n\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.479348 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:22.479348 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479272 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/451a1bd3-fbed-42c5-a04e-45a6befe04f1-config\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.479348 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvm64\" (UniqueName: \"kubernetes.io/projected/5c3d648f-de36-4727-be47-9cb74a3649cb-kube-api-access-kvm64\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:22.479458 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36009ed7-f9b0-48d3-920e-3d6d452d18e8-tmp\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.479458 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479371 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36009ed7-f9b0-48d3-920e-3d6d452d18e8-serving-cert\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.479458 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36009ed7-f9b0-48d3-920e-3d6d452d18e8-service-ca-bundle\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.479458 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479436 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2r97\" (UniqueName: \"kubernetes.io/projected/36009ed7-f9b0-48d3-920e-3d6d452d18e8-kube-api-access-b2r97\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.479588 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.479468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/36009ed7-f9b0-48d3-920e-3d6d452d18e8-snapshots\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.484716 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.484702 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp"] Aug 05 17:20:22.484765 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.484752 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.489015 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.488994 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:20:22.489123 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.489018 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-84tnh\"" Aug 05 17:20:22.489186 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.489163 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Aug 05 17:20:22.490911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.490891 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Aug 05 17:20:22.492934 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.492917 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Aug 05 17:20:22.509182 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.509164 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd"] Aug 05 17:20:22.509271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.509195 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp"] Aug 05 17:20:22.509271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.509262 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" Aug 05 17:20:22.511567 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.511540 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:20:22.511665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.511575 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ld6cj\"" Aug 05 17:20:22.511835 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.511820 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Aug 05 17:20:22.580517 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580492 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/451a1bd3-fbed-42c5-a04e-45a6befe04f1-trusted-ca\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.580682 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/451a1bd3-fbed-42c5-a04e-45a6befe04f1-config\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.580682 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36009ed7-f9b0-48d3-920e-3d6d452d18e8-service-ca-bundle\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.580682 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580666 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/36009ed7-f9b0-48d3-920e-3d6d452d18e8-snapshots\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.580848 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580701 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qfw7\" (UniqueName: \"kubernetes.io/projected/a964634b-ad74-4dd6-9cc1-13361d691cbb-kube-api-access-8qfw7\") pod \"volume-data-source-validator-5b8776669-xkpwp\" (UID: \"a964634b-ad74-4dd6-9cc1-13361d691cbb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" Aug 05 17:20:22.580848 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580734 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/451a1bd3-fbed-42c5-a04e-45a6befe04f1-serving-cert\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.580848 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580762 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:22.580848 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580831 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36009ed7-f9b0-48d3-920e-3d6d452d18e8-trusted-ca-bundle\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:22.580849 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36009ed7-f9b0-48d3-920e-3d6d452d18e8-tmp\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:22.580912 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls podName:5c3d648f-de36-4727-be47-9cb74a3649cb nodeName:}" failed. No retries permitted until 2026-08-05 17:20:23.080894526 +0000 UTC m=+60.482123333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-vj5pf" (UID: "5c3d648f-de36-4727-be47-9cb74a3649cb") : secret "samples-operator-tls" not found Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580940 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36009ed7-f9b0-48d3-920e-3d6d452d18e8-serving-cert\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.580984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe5d030a-f113-49df-b653-5673455bf156-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2r97\" (UniqueName: \"kubernetes.io/projected/36009ed7-f9b0-48d3-920e-3d6d452d18e8-kube-api-access-b2r97\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tz22\" (UniqueName: \"kubernetes.io/projected/fe5d030a-f113-49df-b653-5673455bf156-kube-api-access-5tz22\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.581410 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581101 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe5d030a-f113-49df-b653-5673455bf156-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.581410 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581132 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvm64\" (UniqueName: \"kubernetes.io/projected/5c3d648f-de36-4727-be47-9cb74a3649cb-kube-api-access-kvm64\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:22.581410 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581165 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx22n\" (UniqueName: \"kubernetes.io/projected/451a1bd3-fbed-42c5-a04e-45a6befe04f1-kube-api-access-xx22n\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.581410 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581232 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36009ed7-f9b0-48d3-920e-3d6d452d18e8-service-ca-bundle\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581410 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581239 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36009ed7-f9b0-48d3-920e-3d6d452d18e8-tmp\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581410 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581321 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/451a1bd3-fbed-42c5-a04e-45a6befe04f1-config\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.581726 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581702 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/36009ed7-f9b0-48d3-920e-3d6d452d18e8-snapshots\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.581866 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/451a1bd3-fbed-42c5-a04e-45a6befe04f1-trusted-ca\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.581928 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.581872 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36009ed7-f9b0-48d3-920e-3d6d452d18e8-trusted-ca-bundle\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.584415 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.584394 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36009ed7-f9b0-48d3-920e-3d6d452d18e8-serving-cert\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.584518 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.584415 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/451a1bd3-fbed-42c5-a04e-45a6befe04f1-serving-cert\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.593315 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.593294 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvm64\" (UniqueName: \"kubernetes.io/projected/5c3d648f-de36-4727-be47-9cb74a3649cb-kube-api-access-kvm64\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:22.593399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.593311 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx22n\" (UniqueName: \"kubernetes.io/projected/451a1bd3-fbed-42c5-a04e-45a6befe04f1-kube-api-access-xx22n\") pod \"console-operator-5ccd6b445f-mxs75\" (UID: \"451a1bd3-fbed-42c5-a04e-45a6befe04f1\") " pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.594499 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.594482 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2r97\" (UniqueName: \"kubernetes.io/projected/36009ed7-f9b0-48d3-920e-3d6d452d18e8-kube-api-access-b2r97\") pod \"insights-operator-77696685b-75ch2\" (UID: \"36009ed7-f9b0-48d3-920e-3d6d452d18e8\") " pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.681975 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.681909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe5d030a-f113-49df-b653-5673455bf156-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.681975 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.681947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tz22\" (UniqueName: \"kubernetes.io/projected/fe5d030a-f113-49df-b653-5673455bf156-kube-api-access-5tz22\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.682179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.681978 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe5d030a-f113-49df-b653-5673455bf156-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.682179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.682089 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qfw7\" (UniqueName: \"kubernetes.io/projected/a964634b-ad74-4dd6-9cc1-13361d691cbb-kube-api-access-8qfw7\") pod \"volume-data-source-validator-5b8776669-xkpwp\" (UID: \"a964634b-ad74-4dd6-9cc1-13361d691cbb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" Aug 05 17:20:22.682465 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.682441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe5d030a-f113-49df-b653-5673455bf156-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.683810 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.683789 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe5d030a-f113-49df-b653-5673455bf156-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.689553 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.689529 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tz22\" (UniqueName: \"kubernetes.io/projected/fe5d030a-f113-49df-b653-5673455bf156-kube-api-access-5tz22\") pod \"kube-storage-version-migrator-operator-7878f4b88c-b8lxd\" (UID: \"fe5d030a-f113-49df-b653-5673455bf156\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.689760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.689742 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qfw7\" (UniqueName: \"kubernetes.io/projected/a964634b-ad74-4dd6-9cc1-13361d691cbb-kube-api-access-8qfw7\") pod \"volume-data-source-validator-5b8776669-xkpwp\" (UID: \"a964634b-ad74-4dd6-9cc1-13361d691cbb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" Aug 05 17:20:22.711116 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.711098 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:22.737385 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.737365 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-75ch2" Aug 05 17:20:22.794978 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.794943 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" Aug 05 17:20:22.817507 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.817479 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" Aug 05 17:20:22.891071 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.889173 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-mxs75"] Aug 05 17:20:22.891071 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.890778 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-75ch2"] Aug 05 17:20:22.983243 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.983215 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd"] Aug 05 17:20:22.987078 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:22.987033 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5d030a_f113_49df_b653_5673455bf156.slice/crio-2a0367d8fb46216e1c76335a8f03bb1d7b3a6674a36d6424b835070eb11506ef WatchSource:0}: Error finding container 2a0367d8fb46216e1c76335a8f03bb1d7b3a6674a36d6424b835070eb11506ef: Status 404 returned error can't find the container with id 2a0367d8fb46216e1c76335a8f03bb1d7b3a6674a36d6424b835070eb11506ef Aug 05 17:20:22.998232 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:22.998211 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp"] Aug 05 17:20:23.000615 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:23.000594 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda964634b_ad74_4dd6_9cc1_13361d691cbb.slice/crio-7ca67a55e6955f9b7dfe82ea0392a3d52a3f621290f239c67c58414604476e44 WatchSource:0}: Error finding container 7ca67a55e6955f9b7dfe82ea0392a3d52a3f621290f239c67c58414604476e44: Status 404 returned error can't find the container with id 7ca67a55e6955f9b7dfe82ea0392a3d52a3f621290f239c67c58414604476e44 Aug 05 17:20:23.094709 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:23.094683 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:23.094860 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:23.094819 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 17:20:23.094901 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:23.094879 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls podName:5c3d648f-de36-4727-be47-9cb74a3649cb nodeName:}" failed. No retries permitted until 2026-08-05 17:20:24.094862755 +0000 UTC m=+61.496091547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-vj5pf" (UID: "5c3d648f-de36-4727-be47-9cb74a3649cb") : secret "samples-operator-tls" not found Aug 05 17:20:23.340821 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:23.340786 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" event={"ID":"a964634b-ad74-4dd6-9cc1-13361d691cbb","Type":"ContainerStarted","Data":"7ca67a55e6955f9b7dfe82ea0392a3d52a3f621290f239c67c58414604476e44"} Aug 05 17:20:23.341702 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:23.341681 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" event={"ID":"fe5d030a-f113-49df-b653-5673455bf156","Type":"ContainerStarted","Data":"2a0367d8fb46216e1c76335a8f03bb1d7b3a6674a36d6424b835070eb11506ef"} Aug 05 17:20:23.342677 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:23.342656 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-75ch2" event={"ID":"36009ed7-f9b0-48d3-920e-3d6d452d18e8","Type":"ContainerStarted","Data":"8226d36f9ba32ff44502353ec74b96c74bb301a0609cb9bb1b9fa5e8e80e9e94"} Aug 05 17:20:23.343524 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:23.343500 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" event={"ID":"451a1bd3-fbed-42c5-a04e-45a6befe04f1","Type":"ContainerStarted","Data":"86061dee91cbb7819fc9715330830b82d53469f5bde41c4cfd77d972615e56c4"} Aug 05 17:20:24.105326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:24.105248 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:24.105787 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:24.105416 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 17:20:24.105787 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:24.105489 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls podName:5c3d648f-de36-4727-be47-9cb74a3649cb nodeName:}" failed. No retries permitted until 2026-08-05 17:20:26.105470719 +0000 UTC m=+63.506699535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-vj5pf" (UID: "5c3d648f-de36-4727-be47-9cb74a3649cb") : secret "samples-operator-tls" not found Aug 05 17:20:24.271198 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:24.271174 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d2f2g" Aug 05 17:20:24.347942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:24.347903 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" event={"ID":"a964634b-ad74-4dd6-9cc1-13361d691cbb","Type":"ContainerStarted","Data":"999e80c64d37770054f4e0ec1f4168adee53cf417c747174df2e5daa943ed494"} Aug 05 17:20:24.369370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:24.369284 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-xkpwp" podStartSLOduration=1.158499155 podStartE2EDuration="2.36926674s" totalImagesPullingTime="1.210767585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:24.368487218 +0000 UTC m=+61.769716042" watchObservedRunningTime="2026-08-05 17:20:24.36926674 +0000 UTC m=+61.770495532" Aug 05 17:20:26.126274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.126191 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:26.126656 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:26.126318 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 17:20:26.126656 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:26.126374 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls podName:5c3d648f-de36-4727-be47-9cb74a3649cb nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.126360278 +0000 UTC m=+67.527589071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-vj5pf" (UID: "5c3d648f-de36-4727-be47-9cb74a3649cb") : secret "samples-operator-tls" not found Aug 05 17:20:26.355605 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.355563 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" event={"ID":"fe5d030a-f113-49df-b653-5673455bf156","Type":"ContainerStarted","Data":"07575b7210665f0aff31332984a3b612220990f110ab79559cf9c74fad70ada2"} Aug 05 17:20:26.357103 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.357074 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-75ch2" event={"ID":"36009ed7-f9b0-48d3-920e-3d6d452d18e8","Type":"ContainerStarted","Data":"72218df46390f72b6d5b42eab65b9b63572ba875b5763559bc2b231f58eaa3d8"} Aug 05 17:20:26.358613 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.358594 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/0.log" Aug 05 17:20:26.358715 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.358627 2575 generic.go:356] "Generic (PLEG): container finished" podID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" containerID="69a4a429275c03b2928c49e4ee00de6ecf68ebb866986336404520b3926dcb55" exitCode=255 Aug 05 17:20:26.358715 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.358667 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" event={"ID":"451a1bd3-fbed-42c5-a04e-45a6befe04f1","Type":"ContainerDied","Data":"69a4a429275c03b2928c49e4ee00de6ecf68ebb866986336404520b3926dcb55"} Aug 05 17:20:26.358908 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.358891 2575 scope.go:117] "RemoveContainer" containerID="69a4a429275c03b2928c49e4ee00de6ecf68ebb866986336404520b3926dcb55" Aug 05 17:20:26.372038 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.371996 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" podStartSLOduration=1.533423639 podStartE2EDuration="4.371982831s" totalImagesPullingTime="2.838559192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:26.371322143 +0000 UTC m=+63.772550958" watchObservedRunningTime="2026-08-05 17:20:26.371982831 +0000 UTC m=+63.773211623" Aug 05 17:20:26.408357 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.408320 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-75ch2" podStartSLOduration=1.49282915 podStartE2EDuration="4.408306451s" totalImagesPullingTime="2.915477301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:26.408011935 +0000 UTC m=+63.809240752" watchObservedRunningTime="2026-08-05 17:20:26.408306451 +0000 UTC m=+63.809535284" Aug 05 17:20:26.695639 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.695542 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g69mz"] Aug 05 17:20:26.699537 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.698784 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.701562 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.701376 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 05 17:20:26.701562 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.701447 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qfc7x\"" Aug 05 17:20:26.701562 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.701454 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 05 17:20:26.732152 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.732116 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2whn\" (UniqueName: \"kubernetes.io/projected/56167886-dcec-4258-a9f9-286f063b0887-kube-api-access-x2whn\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.732285 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.732236 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.732285 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.732267 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56167886-dcec-4258-a9f9-286f063b0887-crio-socket\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.732373 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.732297 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56167886-dcec-4258-a9f9-286f063b0887-data-volume\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.732412 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.732366 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.735387 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.735366 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g69mz"] Aug 05 17:20:26.833527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.833493 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.833676 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.833560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2whn\" (UniqueName: \"kubernetes.io/projected/56167886-dcec-4258-a9f9-286f063b0887-kube-api-access-x2whn\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.833676 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.833612 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.833676 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.833631 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56167886-dcec-4258-a9f9-286f063b0887-crio-socket\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.833676 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:26.833631 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 17:20:26.833871 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.833694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56167886-dcec-4258-a9f9-286f063b0887-crio-socket\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.833871 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:26.833694 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 17:20:26.833871 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:26.833702 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.33368718 +0000 UTC m=+64.734915973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : secret "insights-runtime-extractor-tls" not found Aug 05 17:20:26.833871 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:26.833770 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:27.333758648 +0000 UTC m=+64.734987441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : configmap "kube-rbac-proxy" not found Aug 05 17:20:26.833871 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.833850 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56167886-dcec-4258-a9f9-286f063b0887-data-volume\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.834154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.834139 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56167886-dcec-4258-a9f9-286f063b0887-data-volume\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:26.856315 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:26.856289 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2whn\" (UniqueName: \"kubernetes.io/projected/56167886-dcec-4258-a9f9-286f063b0887-kube-api-access-x2whn\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:27.337871 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.337834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:27.338305 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.337894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:27.338305 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.337950 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 17:20:27.338305 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.338019 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:28.338000749 +0000 UTC m=+65.739229541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : configmap "kube-rbac-proxy" not found Aug 05 17:20:27.338305 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.338023 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 17:20:27.338305 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.338085 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:28.338069449 +0000 UTC m=+65.739298255 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : secret "insights-runtime-extractor-tls" not found Aug 05 17:20:27.365615 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.365584 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/1.log" Aug 05 17:20:27.366126 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.366108 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/0.log" Aug 05 17:20:27.366199 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.366150 2575 generic.go:356] "Generic (PLEG): container finished" podID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" containerID="2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980" exitCode=255 Aug 05 17:20:27.366199 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.366180 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" event={"ID":"451a1bd3-fbed-42c5-a04e-45a6befe04f1","Type":"ContainerDied","Data":"2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980"} Aug 05 17:20:27.366306 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.366270 2575 scope.go:117] "RemoveContainer" containerID="69a4a429275c03b2928c49e4ee00de6ecf68ebb866986336404520b3926dcb55" Aug 05 17:20:27.366443 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.366428 2575 scope.go:117] "RemoveContainer" containerID="2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980" Aug 05 17:20:27.366629 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.366604 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-mxs75_openshift-console-operator(451a1bd3-fbed-42c5-a04e-45a6befe04f1)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podUID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" Aug 05 17:20:27.741862 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.741784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:20:27.744008 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.743987 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 05 17:20:27.752693 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.752677 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Aug 05 17:20:27.752785 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:27.752744 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs podName:cd322441-fd87-4da5-babe-874dda982d83 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:31.752723166 +0000 UTC m=+129.153951961 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs") pod "network-metrics-daemon-2m4bd" (UID: "cd322441-fd87-4da5-babe-874dda982d83") : secret "metrics-daemon-secret" not found Aug 05 17:20:27.842736 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.842693 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:20:27.845363 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.845347 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 05 17:20:27.854914 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.854888 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22bf72ca-6129-4438-b017-59817d913812-original-pull-secret\") pod \"global-pull-secret-syncer-dpb7v\" (UID: \"22bf72ca-6129-4438-b017-59817d913812\") " pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:20:27.942502 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.942471 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dpb7v" Aug 05 17:20:27.943753 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.943734 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:20:27.946239 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.946216 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 05 17:20:27.956527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.956499 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 05 17:20:27.966639 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:27.966619 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs7jp\" (UniqueName: \"kubernetes.io/projected/99e75228-e8bf-44e8-9781-1121221d15b1-kube-api-access-gs7jp\") pod \"network-check-target-sgrxg\" (UID: \"99e75228-e8bf-44e8-9781-1121221d15b1\") " pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:20:28.060304 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.060276 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dpb7v"] Aug 05 17:20:28.063375 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:28.063349 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22bf72ca_6129_4438_b017_59817d913812.slice/crio-7ad4e2c4bbeb25f7c16a8d54961a4f9f125777ed817e36069423606eb2e6cf76 WatchSource:0}: Error finding container 7ad4e2c4bbeb25f7c16a8d54961a4f9f125777ed817e36069423606eb2e6cf76: Status 404 returned error can't find the container with id 7ad4e2c4bbeb25f7c16a8d54961a4f9f125777ed817e36069423606eb2e6cf76 Aug 05 17:20:28.232588 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.232557 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bmhgk\"" Aug 05 17:20:28.241466 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.241445 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:20:28.347582 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.347537 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:28.347988 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.347599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:28.347988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:28.347669 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 17:20:28.347988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:28.347732 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 17:20:28.347988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:28.347745 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.347727051 +0000 UTC m=+67.748955844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : configmap "kube-rbac-proxy" not found Aug 05 17:20:28.347988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:28.347774 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:30.347761478 +0000 UTC m=+67.748990271 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : secret "insights-runtime-extractor-tls" not found Aug 05 17:20:28.365864 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.365839 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sgrxg"] Aug 05 17:20:28.369522 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:28.369500 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e75228_e8bf_44e8_9781_1121221d15b1.slice/crio-4c3a19f055810e8102748267f47966d9b92402014a9723f60d89adf910067909 WatchSource:0}: Error finding container 4c3a19f055810e8102748267f47966d9b92402014a9723f60d89adf910067909: Status 404 returned error can't find the container with id 4c3a19f055810e8102748267f47966d9b92402014a9723f60d89adf910067909 Aug 05 17:20:28.370640 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.370623 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/1.log" Aug 05 17:20:28.371003 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.370984 2575 scope.go:117] "RemoveContainer" containerID="2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980" Aug 05 17:20:28.371194 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:28.371173 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-mxs75_openshift-console-operator(451a1bd3-fbed-42c5-a04e-45a6befe04f1)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podUID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" Aug 05 17:20:28.371871 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.371853 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dpb7v" event={"ID":"22bf72ca-6129-4438-b017-59817d913812","Type":"ContainerStarted","Data":"7ad4e2c4bbeb25f7c16a8d54961a4f9f125777ed817e36069423606eb2e6cf76"} Aug 05 17:20:28.653584 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:28.653556 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9gz9l_94edb85c-3031-405a-bf80-76c2d0e4ed88/dns-node-resolver/0.log" Aug 05 17:20:29.377464 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:29.377397 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sgrxg" event={"ID":"99e75228-e8bf-44e8-9781-1121221d15b1","Type":"ContainerStarted","Data":"4c3a19f055810e8102748267f47966d9b92402014a9723f60d89adf910067909"} Aug 05 17:20:29.454390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:29.454361 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9cf2h_838305c7-2d97-492f-bf25-e2f8f485ae5d/node-ca/0.log" Aug 05 17:20:30.165309 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:30.165274 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:30.165525 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.165429 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 05 17:20:30.165525 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.165508 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls podName:5c3d648f-de36-4727-be47-9cb74a3649cb nodeName:}" failed. No retries permitted until 2026-08-05 17:20:38.165489438 +0000 UTC m=+75.566718236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-vj5pf" (UID: "5c3d648f-de36-4727-be47-9cb74a3649cb") : secret "samples-operator-tls" not found Aug 05 17:20:30.367475 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:30.367435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:20:30.367647 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:30.367494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:30.367647 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:30.367556 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:20:30.367647 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367602 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 05 17:20:30.367647 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367623 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765d5c5557-4zdr8: secret "image-registry-tls" not found Aug 05 17:20:30.367647 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367599 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367659 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367668 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:30.367603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367684 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls podName:72739ecf-e267-4cc8-b2f3-be3d56767b3b nodeName:}" failed. No retries permitted until 2026-08-05 17:21:02.3676652 +0000 UTC m=+99.768894019 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls") pod "image-registry-765d5c5557-4zdr8" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b") : secret "image-registry-tls" not found Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367705 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:34.367697159 +0000 UTC m=+71.768925950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : secret "insights-runtime-extractor-tls" not found Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367755 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls podName:b46f48a8-01a7-4ba2-bcc3-b5892a16caf7 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:02.36774705 +0000 UTC m=+99.768975848 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls") pod "dns-default-cb75m" (UID: "b46f48a8-01a7-4ba2-bcc3-b5892a16caf7") : secret "dns-default-metrics-tls" not found Aug 05 17:20:30.367863 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:30.367768 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:34.36776112 +0000 UTC m=+71.768989916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : configmap "kube-rbac-proxy" not found Aug 05 17:20:30.453102 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:30.452994 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p8tz8_298eeb1d-3ed1-407f-990f-08b90ccd84f5/serve-healthcheck-canary/0.log" Aug 05 17:20:32.712070 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:32.712018 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:32.712429 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:32.712119 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:32.712469 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:32.712453 2575 scope.go:117] "RemoveContainer" containerID="2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980" Aug 05 17:20:32.712619 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:32.712598 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-mxs75_openshift-console-operator(451a1bd3-fbed-42c5-a04e-45a6befe04f1)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podUID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" Aug 05 17:20:33.392522 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:33.392489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sgrxg" event={"ID":"99e75228-e8bf-44e8-9781-1121221d15b1","Type":"ContainerStarted","Data":"c11b53bee7b54eb07bd88d184e3152451f70d43fdcab7ad8b9ffbbbf10c94264"} Aug 05 17:20:33.392708 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:33.392544 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:20:33.393833 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:33.393808 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dpb7v" event={"ID":"22bf72ca-6129-4438-b017-59817d913812","Type":"ContainerStarted","Data":"c6545a4a988d6e743a4d768497429751901ceda41bc50013b5dfa053e3522b94"} Aug 05 17:20:33.429748 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:33.429702 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dpb7v" podStartSLOduration=64.935226502 podStartE2EDuration="1m9.429689206s" totalImagesPullingTime="4.494462704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:33.429493872 +0000 UTC m=+70.830722685" watchObservedRunningTime="2026-08-05 17:20:33.429689206 +0000 UTC m=+70.830918021" Aug 05 17:20:33.429859 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:33.429778 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sgrxg" podStartSLOduration=65.244770849 podStartE2EDuration="1m9.429773687s" totalImagesPullingTime="4.185002838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:33.41170088 +0000 UTC m=+70.812929696" watchObservedRunningTime="2026-08-05 17:20:33.429773687 +0000 UTC m=+70.831002553" Aug 05 17:20:34.401075 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:34.401022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:34.401443 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:34.401109 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:34.401443 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:34.401117 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 05 17:20:34.401443 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:34.401172 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:42.401158153 +0000 UTC m=+79.802386946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : configmap "kube-rbac-proxy" not found Aug 05 17:20:34.401443 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:34.401211 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 17:20:34.401443 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:34.401253 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:42.401240829 +0000 UTC m=+79.802469621 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : secret "insights-runtime-extractor-tls" not found Aug 05 17:20:38.236128 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:38.236092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:38.238362 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:38.238340 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c3d648f-de36-4727-be47-9cb74a3649cb-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-vj5pf\" (UID: \"5c3d648f-de36-4727-be47-9cb74a3649cb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:38.325531 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:38.325505 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dkrzv\"" Aug 05 17:20:38.333336 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:38.333319 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" Aug 05 17:20:38.454870 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:38.454838 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf"] Aug 05 17:20:39.415217 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:39.415177 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" event={"ID":"5c3d648f-de36-4727-be47-9cb74a3649cb","Type":"ContainerStarted","Data":"3489edf0ce1f5a5c33c966314ff5c31a03eb6d8ee67b26046a1354e6301214c0"} Aug 05 17:20:40.420130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:40.420088 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" event={"ID":"5c3d648f-de36-4727-be47-9cb74a3649cb","Type":"ContainerStarted","Data":"fccc7ffbe2de9562bad9445d02ceb1341469591b96ed9196f3188f6dfe7de6f5"} Aug 05 17:20:40.420505 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:40.420155 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" event={"ID":"5c3d648f-de36-4727-be47-9cb74a3649cb","Type":"ContainerStarted","Data":"d308a15f850e03702c3b8def67dd25ede0efb0ab993c0048a330cd839e03d5d2"} Aug 05 17:20:40.438797 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:40.438713 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-vj5pf" podStartSLOduration=16.756071409 podStartE2EDuration="18.438697357s" totalImagesPullingTime="1.682625948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:40.437589931 +0000 UTC m=+77.838818745" watchObservedRunningTime="2026-08-05 17:20:40.438697357 +0000 UTC m=+77.839926201" Aug 05 17:20:42.475794 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:42.475756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:42.476278 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:42.475848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:42.476278 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:42.475897 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 05 17:20:42.476278 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:42.475974 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls podName:56167886-dcec-4258-a9f9-286f063b0887 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:58.475957946 +0000 UTC m=+95.877186748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g69mz" (UID: "56167886-dcec-4258-a9f9-286f063b0887") : secret "insights-runtime-extractor-tls" not found Aug 05 17:20:42.476468 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:42.476446 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56167886-dcec-4258-a9f9-286f063b0887-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:43.023082 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.023040 2575 scope.go:117] "RemoveContainer" containerID="2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980" Aug 05 17:20:43.431537 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.431510 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:20:43.431907 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.431892 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/1.log" Aug 05 17:20:43.431964 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.431924 2575 generic.go:356] "Generic (PLEG): container finished" podID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" containerID="fa46cb58b9561bb83388c68036d6deb889af96c849947ad32aa2d0466e89314f" exitCode=255 Aug 05 17:20:43.432005 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.431968 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" event={"ID":"451a1bd3-fbed-42c5-a04e-45a6befe04f1","Type":"ContainerDied","Data":"fa46cb58b9561bb83388c68036d6deb889af96c849947ad32aa2d0466e89314f"} Aug 05 17:20:43.432069 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.432011 2575 scope.go:117] "RemoveContainer" containerID="2b53dd22d660b51a3a688032b75c731f53243b4db572ac1e30e3c5d6587c3980" Aug 05 17:20:43.432344 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:43.432327 2575 scope.go:117] "RemoveContainer" containerID="fa46cb58b9561bb83388c68036d6deb889af96c849947ad32aa2d0466e89314f" Aug 05 17:20:43.432517 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:43.432497 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-mxs75_openshift-console-operator(451a1bd3-fbed-42c5-a04e-45a6befe04f1)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podUID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" Aug 05 17:20:44.436869 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:44.436842 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:20:49.159561 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.159432 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2"] Aug 05 17:20:49.163620 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.163599 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.165908 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.165890 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 05 17:20:49.166839 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.166821 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 05 17:20:49.166891 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.166865 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Aug 05 17:20:49.168775 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.168757 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 05 17:20:49.175866 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.175847 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2"] Aug 05 17:20:49.333211 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.333179 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a5728f95-8e47-43fd-beda-aa23c7887750-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.333393 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.333232 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5728f95-8e47-43fd-beda-aa23c7887750-tmp\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.333518 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.333495 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gsk8\" (UniqueName: \"kubernetes.io/projected/a5728f95-8e47-43fd-beda-aa23c7887750-kube-api-access-8gsk8\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.434705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.434617 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gsk8\" (UniqueName: \"kubernetes.io/projected/a5728f95-8e47-43fd-beda-aa23c7887750-kube-api-access-8gsk8\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.434705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.434672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a5728f95-8e47-43fd-beda-aa23c7887750-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.434705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.434692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5728f95-8e47-43fd-beda-aa23c7887750-tmp\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.435087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.435041 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5728f95-8e47-43fd-beda-aa23c7887750-tmp\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.437276 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.437252 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a5728f95-8e47-43fd-beda-aa23c7887750-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.443003 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.442976 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gsk8\" (UniqueName: \"kubernetes.io/projected/a5728f95-8e47-43fd-beda-aa23c7887750-kube-api-access-8gsk8\") pod \"klusterlet-addon-workmgr-7f469666f4-2qlz2\" (UID: \"a5728f95-8e47-43fd-beda-aa23c7887750\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.473202 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.473175 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:49.597934 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:49.597885 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2"] Aug 05 17:20:49.600369 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:49.600336 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5728f95_8e47_43fd_beda_aa23c7887750.slice/crio-90ac4062967530e669e993400f13925ccedf12e5615e361c34a5b03a4680a71d WatchSource:0}: Error finding container 90ac4062967530e669e993400f13925ccedf12e5615e361c34a5b03a4680a71d: Status 404 returned error can't find the container with id 90ac4062967530e669e993400f13925ccedf12e5615e361c34a5b03a4680a71d Aug 05 17:20:50.456596 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:50.456559 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" event={"ID":"a5728f95-8e47-43fd-beda-aa23c7887750","Type":"ContainerStarted","Data":"90ac4062967530e669e993400f13925ccedf12e5615e361c34a5b03a4680a71d"} Aug 05 17:20:52.711607 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:52.711574 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:52.712037 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:52.711639 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:20:52.712037 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:52.712005 2575 scope.go:117] "RemoveContainer" containerID="fa46cb58b9561bb83388c68036d6deb889af96c849947ad32aa2d0466e89314f" Aug 05 17:20:52.712232 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:52.712211 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-5ccd6b445f-mxs75_openshift-console-operator(451a1bd3-fbed-42c5-a04e-45a6befe04f1)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podUID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" Aug 05 17:20:53.466563 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:53.466526 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" event={"ID":"a5728f95-8e47-43fd-beda-aa23c7887750","Type":"ContainerStarted","Data":"5d249de5681e374cd3a94e729ff5142c0f7ac820e41929a3935ef8b081483724"} Aug 05 17:20:53.466732 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:53.466690 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:53.468384 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:53.468364 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" Aug 05 17:20:53.483800 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:53.483757 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f469666f4-2qlz2" podStartSLOduration=1.047546992 podStartE2EDuration="4.483746603s" totalImagesPullingTime="3.436199611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:53.482498637 +0000 UTC m=+90.883727456" watchObservedRunningTime="2026-08-05 17:20:53.483746603 +0000 UTC m=+90.884975416" Aug 05 17:20:56.758039 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.758005 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl"] Aug 05 17:20:56.761142 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.761126 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:56.764210 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.764177 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 05 17:20:56.764210 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.764182 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-brqpv\"" Aug 05 17:20:56.770474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.770451 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl"] Aug 05 17:20:56.793738 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.793716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/241e6e0c-371e-4d2f-a8c2-77c7fbf2c145-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jf5cl\" (UID: \"241e6e0c-371e-4d2f-a8c2-77c7fbf2c145\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:56.894265 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:56.894216 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/241e6e0c-371e-4d2f-a8c2-77c7fbf2c145-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jf5cl\" (UID: \"241e6e0c-371e-4d2f-a8c2-77c7fbf2c145\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:56.894414 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:56.894356 2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Aug 05 17:20:56.894414 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:20:56.894413 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/241e6e0c-371e-4d2f-a8c2-77c7fbf2c145-tls-certificates podName:241e6e0c-371e-4d2f-a8c2-77c7fbf2c145 nodeName:}" failed. No retries permitted until 2026-08-05 17:20:57.394399746 +0000 UTC m=+94.795628538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/241e6e0c-371e-4d2f-a8c2-77c7fbf2c145-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-jf5cl" (UID: "241e6e0c-371e-4d2f-a8c2-77c7fbf2c145") : secret "prometheus-operator-admission-webhook-tls" not found Aug 05 17:20:57.397811 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:57.397778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/241e6e0c-371e-4d2f-a8c2-77c7fbf2c145-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jf5cl\" (UID: \"241e6e0c-371e-4d2f-a8c2-77c7fbf2c145\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:57.400113 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:57.400080 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/241e6e0c-371e-4d2f-a8c2-77c7fbf2c145-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-jf5cl\" (UID: \"241e6e0c-371e-4d2f-a8c2-77c7fbf2c145\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:57.670509 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:57.670423 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:57.787440 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:57.787411 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl"] Aug 05 17:20:57.790102 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:57.790077 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241e6e0c_371e_4d2f_a8c2_77c7fbf2c145.slice/crio-b3b6fa1091a3e6a3985247816094f01f4b60f171ab97fc51cb2472f036def4d5 WatchSource:0}: Error finding container b3b6fa1091a3e6a3985247816094f01f4b60f171ab97fc51cb2472f036def4d5: Status 404 returned error can't find the container with id b3b6fa1091a3e6a3985247816094f01f4b60f171ab97fc51cb2472f036def4d5 Aug 05 17:20:58.485894 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:58.485858 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" event={"ID":"241e6e0c-371e-4d2f-a8c2-77c7fbf2c145","Type":"ContainerStarted","Data":"b3b6fa1091a3e6a3985247816094f01f4b60f171ab97fc51cb2472f036def4d5"} Aug 05 17:20:58.508354 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:58.508324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:58.510914 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:58.510890 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56167886-dcec-4258-a9f9-286f063b0887-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g69mz\" (UID: \"56167886-dcec-4258-a9f9-286f063b0887\") " pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:58.809660 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:58.809626 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g69mz" Aug 05 17:20:59.073799 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.073726 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g69mz"] Aug 05 17:20:59.076554 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:20:59.076531 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56167886_dcec_4258_a9f9_286f063b0887.slice/crio-4d918e190bf14f580932234f44f1676e300bc19008cb1694af67184fc76d388d WatchSource:0}: Error finding container 4d918e190bf14f580932234f44f1676e300bc19008cb1694af67184fc76d388d: Status 404 returned error can't find the container with id 4d918e190bf14f580932234f44f1676e300bc19008cb1694af67184fc76d388d Aug 05 17:20:59.490689 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.490653 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" event={"ID":"241e6e0c-371e-4d2f-a8c2-77c7fbf2c145","Type":"ContainerStarted","Data":"c73f385086ad8af7c43325f183d99e331204209fd004075798dca98630bc0d35"} Aug 05 17:20:59.490861 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.490714 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:59.492263 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.492240 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g69mz" event={"ID":"56167886-dcec-4258-a9f9-286f063b0887","Type":"ContainerStarted","Data":"8e1ce06664fc947d138df5240af558ed236944a75dbb87a017f24147c11e52fe"} Aug 05 17:20:59.492347 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.492285 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g69mz" event={"ID":"56167886-dcec-4258-a9f9-286f063b0887","Type":"ContainerStarted","Data":"4d918e190bf14f580932234f44f1676e300bc19008cb1694af67184fc76d388d"} Aug 05 17:20:59.495897 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.495881 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" Aug 05 17:20:59.507763 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:20:59.507724 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-jf5cl" podStartSLOduration=1.973282617 podStartE2EDuration="3.5077116s" totalImagesPullingTime="1.534428983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:59.506567282 +0000 UTC m=+96.907796090" watchObservedRunningTime="2026-08-05 17:20:59.5077116 +0000 UTC m=+96.908940413" Aug 05 17:21:00.497109 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:00.497082 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g69mz" event={"ID":"56167886-dcec-4258-a9f9-286f063b0887","Type":"ContainerStarted","Data":"eef5be0c5c2beae0f68f49cff7ebc46b0e56932f803415628b25a19f253ffba0"} Aug 05 17:21:02.440616 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.440527 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:21:02.440616 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.440590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:02.442954 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.442928 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b46f48a8-01a7-4ba2-bcc3-b5892a16caf7-metrics-tls\") pod \"dns-default-cb75m\" (UID: \"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7\") " pod="openshift-dns/dns-default-cb75m" Aug 05 17:21:02.443085 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.442954 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"image-registry-765d5c5557-4zdr8\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:02.506924 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.506884 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g69mz" event={"ID":"56167886-dcec-4258-a9f9-286f063b0887","Type":"ContainerStarted","Data":"4e0c20cf3de2f50d3ecf788bcdef63b6aeeebe97a212d7af1d9adab4f5b50ede"} Aug 05 17:21:02.528337 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.528288 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g69mz" podStartSLOduration=33.64706258 podStartE2EDuration="36.528267043s" totalImagesPullingTime="2.881204463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:20:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:02.528165686 +0000 UTC m=+99.929394498" watchObservedRunningTime="2026-08-05 17:21:02.528267043 +0000 UTC m=+99.929495856" Aug 05 17:21:02.639204 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.639171 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nj4rs\"" Aug 05 17:21:02.647958 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.647932 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:02.664348 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.664326 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6sjj\"" Aug 05 17:21:02.672892 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.672873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cb75m" Aug 05 17:21:02.806068 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.804684 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-765d5c5557-4zdr8"] Aug 05 17:21:02.810845 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:02.810820 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72739ecf_e267_4cc8_b2f3_be3d56767b3b.slice/crio-bb794ede3415085cdfa20afa829ae4f28a9835c3bfd8b3e96322cb1c88845574 WatchSource:0}: Error finding container bb794ede3415085cdfa20afa829ae4f28a9835c3bfd8b3e96322cb1c88845574: Status 404 returned error can't find the container with id bb794ede3415085cdfa20afa829ae4f28a9835c3bfd8b3e96322cb1c88845574 Aug 05 17:21:02.822343 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:02.822322 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cb75m"] Aug 05 17:21:02.824839 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:02.824815 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46f48a8_01a7_4ba2_bcc3_b5892a16caf7.slice/crio-7527ab2d493fd4866ee54d6ba974735c94e85466aa5156904a4b64caf6b2aaae WatchSource:0}: Error finding container 7527ab2d493fd4866ee54d6ba974735c94e85466aa5156904a4b64caf6b2aaae: Status 404 returned error can't find the container with id 7527ab2d493fd4866ee54d6ba974735c94e85466aa5156904a4b64caf6b2aaae Aug 05 17:21:03.511984 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:03.511945 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cb75m" event={"ID":"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7","Type":"ContainerStarted","Data":"7527ab2d493fd4866ee54d6ba974735c94e85466aa5156904a4b64caf6b2aaae"} Aug 05 17:21:03.514136 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:03.514090 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" event={"ID":"72739ecf-e267-4cc8-b2f3-be3d56767b3b","Type":"ContainerStarted","Data":"cffb71e3d1b2eb2392d5e5f1b264e8f8755cfa2f0911520ad3b011614b520edd"} Aug 05 17:21:03.514265 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:03.514141 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" event={"ID":"72739ecf-e267-4cc8-b2f3-be3d56767b3b","Type":"ContainerStarted","Data":"bb794ede3415085cdfa20afa829ae4f28a9835c3bfd8b3e96322cb1c88845574"} Aug 05 17:21:03.514524 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:03.514468 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:03.536995 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:03.536941 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" podStartSLOduration=99.536920808 podStartE2EDuration="1m39.536920808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:03.535083112 +0000 UTC m=+100.936311928" watchObservedRunningTime="2026-08-05 17:21:03.536920808 +0000 UTC m=+100.938149624" Aug 05 17:21:04.171762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.171729 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5"] Aug 05 17:21:04.191094 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.191069 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.195501 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.195020 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Aug 05 17:21:04.195501 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.195117 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 05 17:21:04.195501 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.195347 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 05 17:21:04.195756 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.195528 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6skzd\"" Aug 05 17:21:04.195756 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.195689 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5"] Aug 05 17:21:04.196223 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.196196 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 05 17:21:04.196488 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.196469 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Aug 05 17:21:04.197115 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.197093 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rl6rc"] Aug 05 17:21:04.219001 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.218963 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp"] Aug 05 17:21:04.219165 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.219115 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.221506 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.221481 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 05 17:21:04.221506 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.221497 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kbf9b\"" Aug 05 17:21:04.221682 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.221659 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 05 17:21:04.221783 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.221763 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 05 17:21:04.241148 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.241119 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp"] Aug 05 17:21:04.241282 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.241254 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.244042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.244023 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 05 17:21:04.244179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.244098 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8p667\"" Aug 05 17:21:04.244179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.244103 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 05 17:21:04.244551 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.244534 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 05 17:21:04.254681 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254653 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6982795-4770-45f8-90e8-ead2e4c904e5-metrics-client-ca\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.254792 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-root\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.254792 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254744 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6l5\" (UniqueName: \"kubernetes.io/projected/a6982795-4770-45f8-90e8-ead2e4c904e5-kube-api-access-qk6l5\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.254792 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254769 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd97f\" (UniqueName: \"kubernetes.io/projected/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-kube-api-access-qd97f\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.254942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254802 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.254942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254861 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-wtmp\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.254942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254898 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.254942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-tls\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.255173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.254961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-textfile\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.255173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.255014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.255173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.255043 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.255173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.255110 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-sys\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.255173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.255154 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356582 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-root\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6l5\" (UniqueName: \"kubernetes.io/projected/a6982795-4770-45f8-90e8-ead2e4c904e5-kube-api-access-qk6l5\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356618 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd97f\" (UniqueName: \"kubernetes.io/projected/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-kube-api-access-qd97f\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.356760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356666 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.356760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356671 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-root\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356686 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.356760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356727 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-wtmp\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356759 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:21:04.356769 2575 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356790 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-tls\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:21:04.356826 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls podName:ee5b1533-ac9b-404e-bf8a-920d08e2c0bb nodeName:}" failed. No retries permitted until 2026-08-05 17:21:04.856806733 +0000 UTC m=+102.258035527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls") pod "openshift-state-metrics-6bb4d7b4dd-ljhd5" (UID: "ee5b1533-ac9b-404e-bf8a-920d08e2c0bb") : secret "openshift-state-metrics-tls" not found Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356844 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:21:04.356878 2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:21:04.356928 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-tls podName:a6982795-4770-45f8-90e8-ead2e4c904e5 nodeName:}" failed. No retries permitted until 2026-08-05 17:21:04.856911116 +0000 UTC m=+102.258139922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-tls") pod "node-exporter-rl6rc" (UID: "a6982795-4770-45f8-90e8-ead2e4c904e5") : secret "node-exporter-tls" not found Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-textfile\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.356966 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356924 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-wtmp\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.356982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/96c12aea-18c7-45cc-979f-f15b9b968020-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357096 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-sys\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357162 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357194 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c12aea-18c7-45cc-979f-f15b9b968020-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357201 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-textfile\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357407 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357217 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6982795-4770-45f8-90e8-ead2e4c904e5-metrics-client-ca\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357879 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357855 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6982795-4770-45f8-90e8-ead2e4c904e5-metrics-client-ca\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.357879 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357870 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.358000 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357907 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.358000 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd29q\" (UniqueName: \"kubernetes.io/projected/96c12aea-18c7-45cc-979f-f15b9b968020-kube-api-access-rd29q\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.358000 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.357921 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6982795-4770-45f8-90e8-ead2e4c904e5-sys\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.359701 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.359669 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.359701 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.359683 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.365937 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.365916 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd97f\" (UniqueName: \"kubernetes.io/projected/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-kube-api-access-qd97f\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.367334 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.367310 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6l5\" (UniqueName: \"kubernetes.io/projected/a6982795-4770-45f8-90e8-ead2e4c904e5-kube-api-access-qk6l5\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.399229 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.399195 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sgrxg" Aug 05 17:21:04.458658 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.458580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.458658 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.458619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.458658 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.458642 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/96c12aea-18c7-45cc-979f-f15b9b968020-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.458894 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.458679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c12aea-18c7-45cc-979f-f15b9b968020-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.458894 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.458698 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd29q\" (UniqueName: \"kubernetes.io/projected/96c12aea-18c7-45cc-979f-f15b9b968020-kube-api-access-rd29q\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.459136 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.459111 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.459704 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.459432 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/96c12aea-18c7-45cc-979f-f15b9b968020-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.460084 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.459874 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.464802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.461826 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.464802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.462283 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96c12aea-18c7-45cc-979f-f15b9b968020-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.464960 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.464802 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96c12aea-18c7-45cc-979f-f15b9b968020-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.467546 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.467524 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd29q\" (UniqueName: \"kubernetes.io/projected/96c12aea-18c7-45cc-979f-f15b9b968020-kube-api-access-rd29q\") pod \"kube-state-metrics-6bd9664c4-g64gp\" (UID: \"96c12aea-18c7-45cc-979f-f15b9b968020\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.550824 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.550792 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" Aug 05 17:21:04.697201 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.697180 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp"] Aug 05 17:21:04.699595 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:04.699568 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c12aea_18c7_45cc_979f_f15b9b968020.slice/crio-e3a6a7f4d42cc05edc770e319b42f1d013e1f2c05cdf4db832d54e1a14355bff WatchSource:0}: Error finding container e3a6a7f4d42cc05edc770e319b42f1d013e1f2c05cdf4db832d54e1a14355bff: Status 404 returned error can't find the container with id e3a6a7f4d42cc05edc770e319b42f1d013e1f2c05cdf4db832d54e1a14355bff Aug 05 17:21:04.861934 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.861905 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:04.862130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.861941 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-tls\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:04.862130 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:21:04.862069 2575 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Aug 05 17:21:04.862254 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:21:04.862163 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls podName:ee5b1533-ac9b-404e-bf8a-920d08e2c0bb nodeName:}" failed. No retries permitted until 2026-08-05 17:21:05.86213682 +0000 UTC m=+103.263365614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls") pod "openshift-state-metrics-6bb4d7b4dd-ljhd5" (UID: "ee5b1533-ac9b-404e-bf8a-920d08e2c0bb") : secret "openshift-state-metrics-tls" not found Aug 05 17:21:04.864097 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:04.864082 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6982795-4770-45f8-90e8-ead2e4c904e5-node-exporter-tls\") pod \"node-exporter-rl6rc\" (UID: \"a6982795-4770-45f8-90e8-ead2e4c904e5\") " pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:05.131716 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.131584 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rl6rc" Aug 05 17:21:05.145473 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:05.145436 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6982795_4770_45f8_90e8_ead2e4c904e5.slice/crio-a1814b9020f1a74722679f8488dfcda9432944246b8cb73e646dca0e08dfb061 WatchSource:0}: Error finding container a1814b9020f1a74722679f8488dfcda9432944246b8cb73e646dca0e08dfb061: Status 404 returned error can't find the container with id a1814b9020f1a74722679f8488dfcda9432944246b8cb73e646dca0e08dfb061 Aug 05 17:21:05.522863 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.522776 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" event={"ID":"96c12aea-18c7-45cc-979f-f15b9b968020","Type":"ContainerStarted","Data":"e3a6a7f4d42cc05edc770e319b42f1d013e1f2c05cdf4db832d54e1a14355bff"} Aug 05 17:21:05.524819 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.524790 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cb75m" event={"ID":"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7","Type":"ContainerStarted","Data":"d39f540caf2d7051a78529b3844e5821e6764a6bd8b4d87fe0d443ef86174038"} Aug 05 17:21:05.524957 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.524839 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cb75m" event={"ID":"b46f48a8-01a7-4ba2-bcc3-b5892a16caf7","Type":"ContainerStarted","Data":"dcfca8a72e912dd7cae5be1766048670aa3f36741c082268c0173078f0e366dd"} Aug 05 17:21:05.525010 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.524984 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cb75m" Aug 05 17:21:05.526236 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.526204 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rl6rc" event={"ID":"a6982795-4770-45f8-90e8-ead2e4c904e5","Type":"ContainerStarted","Data":"a1814b9020f1a74722679f8488dfcda9432944246b8cb73e646dca0e08dfb061"} Aug 05 17:21:05.546495 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.546447 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cb75m" podStartSLOduration=65.44427341 podStartE2EDuration="1m7.546429111s" totalImagesPullingTime="2.102155701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:05.546302459 +0000 UTC m=+102.947531275" watchObservedRunningTime="2026-08-05 17:21:05.546429111 +0000 UTC m=+102.947657924" Aug 05 17:21:05.872802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.872681 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:05.877482 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:05.877415 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee5b1533-ac9b-404e-bf8a-920d08e2c0bb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-ljhd5\" (UID: \"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:06.002964 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.002907 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" Aug 05 17:21:06.161437 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.161405 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5"] Aug 05 17:21:06.177635 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.177601 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d496b45bc-thc85"] Aug 05 17:21:06.180885 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.180864 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.184543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.184444 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 05 17:21:06.184543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.184471 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 05 17:21:06.184543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.184477 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-23jck5lf06onp\"" Aug 05 17:21:06.184543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.184533 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fxg8g\"" Aug 05 17:21:06.184883 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.184856 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 05 17:21:06.185154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.185086 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 05 17:21:06.185287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.185269 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 05 17:21:06.194300 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.194282 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d496b45bc-thc85"] Aug 05 17:21:06.276630 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276588 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-tls\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.276792 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276642 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.276792 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276750 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-grpc-tls\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.276916 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276790 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vqsk\" (UniqueName: \"kubernetes.io/projected/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-kube-api-access-7vqsk\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.276916 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276827 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.276916 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-metrics-client-ca\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.276916 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.277147 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.276947 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.309965 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:06.309930 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5b1533_ac9b_404e_bf8a_920d08e2c0bb.slice/crio-617d08fffe8e53740290417824e496998dfa8d22b2f4b2fbc5fb39e7f31ba1cf WatchSource:0}: Error finding container 617d08fffe8e53740290417824e496998dfa8d22b2f4b2fbc5fb39e7f31ba1cf: Status 404 returned error can't find the container with id 617d08fffe8e53740290417824e496998dfa8d22b2f4b2fbc5fb39e7f31ba1cf Aug 05 17:21:06.378412 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378381 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-tls\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-grpc-tls\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378500 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vqsk\" (UniqueName: \"kubernetes.io/projected/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-kube-api-access-7vqsk\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378734 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378556 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-metrics-client-ca\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378734 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378597 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.378734 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.378643 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.380032 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.380006 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-metrics-client-ca\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.383042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.382074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.383042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.382329 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-grpc-tls\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.383042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.382357 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-tls\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.383042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.382388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.383042 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.382678 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.383604 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.383538 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.390973 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.390953 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vqsk\" (UniqueName: \"kubernetes.io/projected/e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396-kube-api-access-7vqsk\") pod \"thanos-querier-7d496b45bc-thc85\" (UID: \"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396\") " pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.491092 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.491067 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:06.532100 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.532045 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" event={"ID":"96c12aea-18c7-45cc-979f-f15b9b968020","Type":"ContainerStarted","Data":"6dc272d23ecb30c6cb3a19e280357f6e4822449976d9e196891f0752594a942a"} Aug 05 17:21:06.532293 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.532130 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" event={"ID":"96c12aea-18c7-45cc-979f-f15b9b968020","Type":"ContainerStarted","Data":"5c0899112a7def2e3605674827a9251034f0a40e1db28a4a049876362c65cee9"} Aug 05 17:21:06.532293 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.532157 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" event={"ID":"96c12aea-18c7-45cc-979f-f15b9b968020","Type":"ContainerStarted","Data":"d8147195076bc0418c197e35cf67da2b6d29f64394dc2e1082bc19c1834c9c63"} Aug 05 17:21:06.534172 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.534144 2575 generic.go:356] "Generic (PLEG): container finished" podID="a6982795-4770-45f8-90e8-ead2e4c904e5" containerID="3421f4d0df3b0504b90d94cf01799d1ad1451761bacc2934406a6e8b216d2642" exitCode=0 Aug 05 17:21:06.534312 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.534221 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rl6rc" event={"ID":"a6982795-4770-45f8-90e8-ead2e4c904e5","Type":"ContainerDied","Data":"3421f4d0df3b0504b90d94cf01799d1ad1451761bacc2934406a6e8b216d2642"} Aug 05 17:21:06.538109 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.538046 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" event={"ID":"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb","Type":"ContainerStarted","Data":"969ac5eb0fbf8143b790313b8bee6b332212d8b9e2945010b29755661f895b38"} Aug 05 17:21:06.538191 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.538154 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" event={"ID":"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb","Type":"ContainerStarted","Data":"e84fe3f27ed01b1f28b87e789cfca049840d3ca128b1610d28cf058409e8715b"} Aug 05 17:21:06.538343 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.538324 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" event={"ID":"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb","Type":"ContainerStarted","Data":"617d08fffe8e53740290417824e496998dfa8d22b2f4b2fbc5fb39e7f31ba1cf"} Aug 05 17:21:06.552770 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.552327 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-g64gp" podStartSLOduration=1.492874078 podStartE2EDuration="2.552309805s" totalImagesPullingTime="1.059435727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:06.550980303 +0000 UTC m=+103.952209123" watchObservedRunningTime="2026-08-05 17:21:06.552309805 +0000 UTC m=+103.953538618" Aug 05 17:21:06.630514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:06.630486 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d496b45bc-thc85"] Aug 05 17:21:06.633476 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:06.633445 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85cee89_f2f7_4c6e_b7d1_9f3ba1f83396.slice/crio-1b7d5e0323e19e66a63f2bfead2f713ed031c815b9277eabb6275b2cbca1f08c WatchSource:0}: Error finding container 1b7d5e0323e19e66a63f2bfead2f713ed031c815b9277eabb6275b2cbca1f08c: Status 404 returned error can't find the container with id 1b7d5e0323e19e66a63f2bfead2f713ed031c815b9277eabb6275b2cbca1f08c Aug 05 17:21:07.545971 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:07.545934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rl6rc" event={"ID":"a6982795-4770-45f8-90e8-ead2e4c904e5","Type":"ContainerStarted","Data":"f016f50ecaaf8dae9571451f91feba800863bcef9177045b8a46ae28e82b71e9"} Aug 05 17:21:07.546425 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:07.546001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rl6rc" event={"ID":"a6982795-4770-45f8-90e8-ead2e4c904e5","Type":"ContainerStarted","Data":"8616280e07cc123eb4e0bcfc4c31630416c4e71fb25299446267d797990e53b6"} Aug 05 17:21:07.548433 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:07.548397 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" event={"ID":"ee5b1533-ac9b-404e-bf8a-920d08e2c0bb","Type":"ContainerStarted","Data":"244821ed08b342e46b2b0b8bbd064e744d6f30bbe526effa649f3a030f5fc59e"} Aug 05 17:21:07.549712 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:07.549686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"1b7d5e0323e19e66a63f2bfead2f713ed031c815b9277eabb6275b2cbca1f08c"} Aug 05 17:21:07.573589 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:07.573527 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rl6rc" podStartSLOduration=2.322261676 podStartE2EDuration="3.573509119s" totalImagesPullingTime="1.185612635s" totalInitContainerRuntime="65.634808ms" isStatefulPod=true podCreationTimestamp="2026-08-05 17:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:07.572767458 +0000 UTC m=+104.973996286" watchObservedRunningTime="2026-08-05 17:21:07.573509119 +0000 UTC m=+104.974737933" Aug 05 17:21:07.596101 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:07.595992 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-ljhd5" podStartSLOduration=2.728632969 podStartE2EDuration="3.595975294s" totalImagesPullingTime="867.342325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:07.595631089 +0000 UTC m=+104.996859903" watchObservedRunningTime="2026-08-05 17:21:07.595975294 +0000 UTC m=+104.997204105" Aug 05 17:21:08.020477 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.020405 2575 scope.go:117] "RemoveContainer" containerID="fa46cb58b9561bb83388c68036d6deb889af96c849947ad32aa2d0466e89314f" Aug 05 17:21:08.557085 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.557035 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:21:08.557422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.557173 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" event={"ID":"451a1bd3-fbed-42c5-a04e-45a6befe04f1","Type":"ContainerStarted","Data":"b220afdb057459a757771e2f56a0fc8584060948f302f05a397910c8c0705e89"} Aug 05 17:21:08.557503 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.557480 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:21:08.559154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.559067 2575 patch_prober.go:28] interesting pod/console-operator-5ccd6b445f-mxs75 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" start-of-body= Aug 05 17:21:08.559154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.559076 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"429ef319948ae40d872b667de62683c67c913c57d417b488fa5617f85472dd2b"} Aug 05 17:21:08.559154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.559114 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podUID="451a1bd3-fbed-42c5-a04e-45a6befe04f1" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" Aug 05 17:21:08.575322 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.574998 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" podStartSLOduration=43.665470859 podStartE2EDuration="46.574977652s" totalImagesPullingTime="2.909506793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:08.574623239 +0000 UTC m=+105.975852052" watchObservedRunningTime="2026-08-05 17:21:08.574977652 +0000 UTC m=+105.976206466" Aug 05 17:21:08.711743 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.711706 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-d54cd6548-xbhpt"] Aug 05 17:21:08.714830 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.714769 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.717031 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.717007 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5hns5\"" Aug 05 17:21:08.717180 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.717029 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 05 17:21:08.717180 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.717129 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ffiifng6ffap6\"" Aug 05 17:21:08.717286 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.717276 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 05 17:21:08.717401 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.717384 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 05 17:21:08.717461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.717400 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 05 17:21:08.727552 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.727525 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d54cd6548-xbhpt"] Aug 05 17:21:08.803835 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.803792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-759zc\" (UniqueName: \"kubernetes.io/projected/257c259a-c957-4373-a341-339925119047-kube-api-access-759zc\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.803993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.803846 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/257c259a-c957-4373-a341-339925119047-metrics-server-audit-profiles\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.803993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.803895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/257c259a-c957-4373-a341-339925119047-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.803993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.803940 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-secret-metrics-server-client-certs\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.803993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.803960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-client-ca-bundle\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.803993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.803989 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-secret-metrics-server-tls\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.804217 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.804114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/257c259a-c957-4373-a341-339925119047-audit-log\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905203 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905159 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/257c259a-c957-4373-a341-339925119047-metrics-server-audit-profiles\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905398 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/257c259a-c957-4373-a341-339925119047-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905398 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-secret-metrics-server-client-certs\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905398 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905346 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-client-ca-bundle\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905595 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905414 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-secret-metrics-server-tls\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905595 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/257c259a-c957-4373-a341-339925119047-audit-log\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.905700 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-759zc\" (UniqueName: \"kubernetes.io/projected/257c259a-c957-4373-a341-339925119047-kube-api-access-759zc\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.906182 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.905956 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/257c259a-c957-4373-a341-339925119047-audit-log\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.906404 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.906378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/257c259a-c957-4373-a341-339925119047-metrics-server-audit-profiles\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.906563 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.906398 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/257c259a-c957-4373-a341-339925119047-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.908226 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.908202 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-client-ca-bundle\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.908325 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.908247 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-secret-metrics-server-client-certs\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.908449 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.908426 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/257c259a-c957-4373-a341-339925119047-secret-metrics-server-tls\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:08.932167 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:08.932141 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-759zc\" (UniqueName: \"kubernetes.io/projected/257c259a-c957-4373-a341-339925119047-kube-api-access-759zc\") pod \"metrics-server-d54cd6548-xbhpt\" (UID: \"257c259a-c957-4373-a341-339925119047\") " pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:09.024008 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.023907 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:09.189927 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.189899 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-d54cd6548-xbhpt"] Aug 05 17:21:09.192601 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:09.192576 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257c259a_c957_4373_a341_339925119047.slice/crio-fa95f631459667dc961c651b53cb613867daf5677283649289340aff7a0b4359 WatchSource:0}: Error finding container fa95f631459667dc961c651b53cb613867daf5677283649289340aff7a0b4359: Status 404 returned error can't find the container with id fa95f631459667dc961c651b53cb613867daf5677283649289340aff7a0b4359 Aug 05 17:21:09.464649 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.464610 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8d66bd447-22gvw"] Aug 05 17:21:09.467929 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.467908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.470082 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.470046 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v49n2\"" Aug 05 17:21:09.470909 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.470891 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Aug 05 17:21:09.470909 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.470896 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Aug 05 17:21:09.471666 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.471650 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Aug 05 17:21:09.472364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.472347 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Aug 05 17:21:09.472464 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.472364 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Aug 05 17:21:09.476020 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.476001 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Aug 05 17:21:09.489128 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.489108 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8d66bd447-22gvw"] Aug 05 17:21:09.511280 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511252 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-federate-client-tls\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511395 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511305 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-secret-telemeter-client\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511395 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-metrics-client-ca\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511395 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511369 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/ff3aeea5-e44e-452a-8b3b-be1330ffa857-kube-api-access-bk5hr\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-serving-certs-ca-bundle\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-telemeter-client-tls\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.511527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.511502 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.566498 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.566464 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"806f765a42f27fc2064fff2b11729c99689e5fb224997adbb00d38c1b39afd34"} Aug 05 17:21:09.566894 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.566551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"291cce1aa1a244cdc6a3439ab268d1e26c8313452b7f8c1ceafbf8d596c94762"} Aug 05 17:21:09.566894 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.566593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"b04f4a7aeb688ecab0338ac3d0129381a5cabfc3d5bd574a487dc28a8e02aac2"} Aug 05 17:21:09.569195 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.569163 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" event={"ID":"257c259a-c957-4373-a341-339925119047","Type":"ContainerStarted","Data":"fa95f631459667dc961c651b53cb613867daf5677283649289340aff7a0b4359"} Aug 05 17:21:09.574418 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.574399 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-mxs75" Aug 05 17:21:09.612331 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-serving-certs-ca-bundle\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-telemeter-client-tls\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612361 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612458 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-federate-client-tls\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-secret-telemeter-client\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-metrics-client-ca\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.612665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.612608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/ff3aeea5-e44e-452a-8b3b-be1330ffa857-kube-api-access-bk5hr\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.614500 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.613898 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-metrics-client-ca\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.614500 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.613948 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.615019 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.614995 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3aeea5-e44e-452a-8b3b-be1330ffa857-serving-certs-ca-bundle\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.616686 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.616660 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.617169 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.617145 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-federate-client-tls\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.617428 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.617411 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-telemeter-client-tls\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.617496 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.617410 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ff3aeea5-e44e-452a-8b3b-be1330ffa857-secret-telemeter-client\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.633722 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.633691 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk5hr\" (UniqueName: \"kubernetes.io/projected/ff3aeea5-e44e-452a-8b3b-be1330ffa857-kube-api-access-bk5hr\") pod \"telemeter-client-8d66bd447-22gvw\" (UID: \"ff3aeea5-e44e-452a-8b3b-be1330ffa857\") " pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.788256 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.788182 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" Aug 05 17:21:09.954915 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:09.954887 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8d66bd447-22gvw"] Aug 05 17:21:09.957083 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:09.957038 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3aeea5_e44e_452a_8b3b_be1330ffa857.slice/crio-eb25149b9feabe39188dbc09593ebaddbb4a03be4a1c0ebeeaaeae32fe289db2 WatchSource:0}: Error finding container eb25149b9feabe39188dbc09593ebaddbb4a03be4a1c0ebeeaaeae32fe289db2: Status 404 returned error can't find the container with id eb25149b9feabe39188dbc09593ebaddbb4a03be4a1c0ebeeaaeae32fe289db2 Aug 05 17:21:10.576274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:10.576246 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"b7b70efbfaa691af9f73607120f2d169ef02e45cc3a7e37b2718749d9f2141e8"} Aug 05 17:21:10.576730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:10.576598 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" event={"ID":"e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396","Type":"ContainerStarted","Data":"6ab40e88499d3ba51843cceaf59f13b591b8b63dc85e6cd8dfe1eec7b913c2dd"} Aug 05 17:21:10.581295 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:10.576916 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:10.582353 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:10.582321 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" event={"ID":"ff3aeea5-e44e-452a-8b3b-be1330ffa857","Type":"ContainerStarted","Data":"eb25149b9feabe39188dbc09593ebaddbb4a03be4a1c0ebeeaaeae32fe289db2"} Aug 05 17:21:10.607034 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:10.606983 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" podStartSLOduration=1.941514934 podStartE2EDuration="4.606963514s" totalImagesPullingTime="2.66544858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:10.605903031 +0000 UTC m=+108.007131844" watchObservedRunningTime="2026-08-05 17:21:10.606963514 +0000 UTC m=+108.008192329" Aug 05 17:21:11.510152 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:11.510120 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-765d5c5557-4zdr8"] Aug 05 17:21:11.589091 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:11.589035 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" event={"ID":"257c259a-c957-4373-a341-339925119047","Type":"ContainerStarted","Data":"14ca076b2c36b308d7c94f73f776ed32c25410a25ebcc5a2c2ae0048680bbffc"} Aug 05 17:21:11.626547 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:11.626489 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" podStartSLOduration=2.272923499 podStartE2EDuration="3.626467155s" totalImagesPullingTime="1.353543656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:11.625892873 +0000 UTC m=+109.027121690" watchObservedRunningTime="2026-08-05 17:21:11.626467155 +0000 UTC m=+109.027695971" Aug 05 17:21:12.596069 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:12.596013 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" event={"ID":"ff3aeea5-e44e-452a-8b3b-be1330ffa857","Type":"ContainerStarted","Data":"5247c4d09c13f95acbd323bd1f6fe50aa018199ec96df1274a19447a41cef5f3"} Aug 05 17:21:13.601586 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:13.601542 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" event={"ID":"ff3aeea5-e44e-452a-8b3b-be1330ffa857","Type":"ContainerStarted","Data":"cf99b256ccf5b6c2143ac8fc08487433731d1e57391253d453aec6be4e6d8fdf"} Aug 05 17:21:13.602087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:13.601604 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" event={"ID":"ff3aeea5-e44e-452a-8b3b-be1330ffa857","Type":"ContainerStarted","Data":"b892a40d4aef4d063eb4c3c1141faa5118325ac61af05bbd279d1a4db934a0ab"} Aug 05 17:21:13.625992 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:13.625940 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8d66bd447-22gvw" podStartSLOduration=1.76801027 podStartE2EDuration="4.625922737s" totalImagesPullingTime="2.857912467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:13.62448863 +0000 UTC m=+111.025726087" watchObservedRunningTime="2026-08-05 17:21:13.625922737 +0000 UTC m=+111.027151551" Aug 05 17:21:15.541096 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:15.541043 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cb75m" Aug 05 17:21:16.594832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:16.594805 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d496b45bc-thc85" Aug 05 17:21:21.144225 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.144190 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-766d686f4b-jh5fz"] Aug 05 17:21:21.150541 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.150510 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.152873 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.152848 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 05 17:21:21.152981 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.152921 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 05 17:21:21.153568 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.153533 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 05 17:21:21.153568 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.153543 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 05 17:21:21.153758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.153567 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 05 17:21:21.153758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.153586 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 05 17:21:21.153758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.153605 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 05 17:21:21.153758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.153690 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sq4kh\"" Aug 05 17:21:21.157291 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.157269 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766d686f4b-jh5fz"] Aug 05 17:21:21.319774 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.319736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-console-config\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.319945 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.319781 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-serving-cert\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.319945 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.319801 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkl9h\" (UniqueName: \"kubernetes.io/projected/6299984f-379e-4258-afae-886df0235139-kube-api-access-rkl9h\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.319945 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.319885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-oauth-serving-cert\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.319945 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.319936 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-oauth-config\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.320100 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.319956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-service-ca\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.421199 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421119 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-console-config\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.421199 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-serving-cert\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.421370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl9h\" (UniqueName: \"kubernetes.io/projected/6299984f-379e-4258-afae-886df0235139-kube-api-access-rkl9h\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.421370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421263 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-oauth-serving-cert\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.421370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421329 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-oauth-config\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.421370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421356 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-service-ca\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.422000 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421979 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-oauth-serving-cert\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.422156 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.421985 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-console-config\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.422156 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.422014 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-service-ca\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.423619 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.423602 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-oauth-config\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.423705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.423686 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-serving-cert\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.431003 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.430984 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkl9h\" (UniqueName: \"kubernetes.io/projected/6299984f-379e-4258-afae-886df0235139-kube-api-access-rkl9h\") pod \"console-766d686f4b-jh5fz\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.460531 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.460504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:21.517091 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.517035 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:21.593383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.593350 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766d686f4b-jh5fz"] Aug 05 17:21:21.596809 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:21.596774 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6299984f_379e_4258_afae_886df0235139.slice/crio-9759231a7ca2766309a557fb84740a7b9e12c1b58a392e64d0c63fd3f0b323ce WatchSource:0}: Error finding container 9759231a7ca2766309a557fb84740a7b9e12c1b58a392e64d0c63fd3f0b323ce: Status 404 returned error can't find the container with id 9759231a7ca2766309a557fb84740a7b9e12c1b58a392e64d0c63fd3f0b323ce Aug 05 17:21:21.633242 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:21.633209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766d686f4b-jh5fz" event={"ID":"6299984f-379e-4258-afae-886df0235139","Type":"ContainerStarted","Data":"9759231a7ca2766309a557fb84740a7b9e12c1b58a392e64d0c63fd3f0b323ce"} Aug 05 17:21:24.647400 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:24.647307 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766d686f4b-jh5fz" event={"ID":"6299984f-379e-4258-afae-886df0235139","Type":"ContainerStarted","Data":"7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222"} Aug 05 17:21:24.666115 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:24.666045 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-766d686f4b-jh5fz" podStartSLOduration=0.899623664 podStartE2EDuration="3.666029147s" totalImagesPullingTime="2.766405483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:24.664617567 +0000 UTC m=+122.065846383" watchObservedRunningTime="2026-08-05 17:21:24.666029147 +0000 UTC m=+122.067257962" Aug 05 17:21:29.025691 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.025659 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:29.026157 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.025709 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:29.144534 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.144501 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797c79c75b-7qsjx"] Aug 05 17:21:29.147940 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.147923 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.157599 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.157564 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 05 17:21:29.162086 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.162060 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797c79c75b-7qsjx"] Aug 05 17:21:29.297463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297386 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-serving-cert\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.297463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297427 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-service-ca\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.297463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297456 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-config\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.297693 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297563 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-oauth-config\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.297693 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297592 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2jl\" (UniqueName: \"kubernetes.io/projected/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-kube-api-access-9m2jl\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.297693 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-trusted-ca-bundle\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.297693 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.297676 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-oauth-serving-cert\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398403 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398364 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-trusted-ca-bundle\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398403 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-oauth-serving-cert\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-serving-cert\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398459 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-service-ca\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-config\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-oauth-config\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.398654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.398557 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2jl\" (UniqueName: \"kubernetes.io/projected/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-kube-api-access-9m2jl\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.399308 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.399284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-config\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.399418 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.399285 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-oauth-serving-cert\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.399418 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.399324 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-service-ca\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.399418 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.399376 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-trusted-ca-bundle\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.401371 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.401342 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-oauth-config\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.401472 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.401455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-serving-cert\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.406176 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.406152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2jl\" (UniqueName: \"kubernetes.io/projected/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-kube-api-access-9m2jl\") pod \"console-797c79c75b-7qsjx\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.461221 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.461186 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:29.592787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.592767 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797c79c75b-7qsjx"] Aug 05 17:21:29.595433 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:29.595401 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd814358_3df6_4d1f_bcf7_8efb6a1db3f3.slice/crio-b7f3004da910e486ef1a939d706e663373e9ceaeb8e2b7291df392ed5034465e WatchSource:0}: Error finding container b7f3004da910e486ef1a939d706e663373e9ceaeb8e2b7291df392ed5034465e: Status 404 returned error can't find the container with id b7f3004da910e486ef1a939d706e663373e9ceaeb8e2b7291df392ed5034465e Aug 05 17:21:29.668340 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.668309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c79c75b-7qsjx" event={"ID":"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3","Type":"ContainerStarted","Data":"4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee"} Aug 05 17:21:29.668493 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.668355 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c79c75b-7qsjx" event={"ID":"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3","Type":"ContainerStarted","Data":"b7f3004da910e486ef1a939d706e663373e9ceaeb8e2b7291df392ed5034465e"} Aug 05 17:21:29.686562 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:29.686502 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797c79c75b-7qsjx" podStartSLOduration=0.686477897 podStartE2EDuration="686.477897ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:29.684810164 +0000 UTC m=+127.086039000" watchObservedRunningTime="2026-08-05 17:21:29.686477897 +0000 UTC m=+127.087706712" Aug 05 17:21:31.461570 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.461539 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:31.462015 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.461613 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:31.466700 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.466675 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:31.679725 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.679698 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:21:31.820855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.820824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:21:31.823072 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.823035 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd322441-fd87-4da5-babe-874dda982d83-metrics-certs\") pod \"network-metrics-daemon-2m4bd\" (UID: \"cd322441-fd87-4da5-babe-874dda982d83\") " pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:21:31.838754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.838735 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4rdxm\"" Aug 05 17:21:31.847399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.847380 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2m4bd" Aug 05 17:21:31.987969 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:31.987936 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2m4bd"] Aug 05 17:21:31.990997 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:21:31.990968 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd322441_fd87_4da5_babe_874dda982d83.slice/crio-0219089dacf1020a472035dfd33ea4bb89ae79dee5cf3cf39feed761160e4797 WatchSource:0}: Error finding container 0219089dacf1020a472035dfd33ea4bb89ae79dee5cf3cf39feed761160e4797: Status 404 returned error can't find the container with id 0219089dacf1020a472035dfd33ea4bb89ae79dee5cf3cf39feed761160e4797 Aug 05 17:21:32.681370 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:32.681328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2m4bd" event={"ID":"cd322441-fd87-4da5-babe-874dda982d83","Type":"ContainerStarted","Data":"0219089dacf1020a472035dfd33ea4bb89ae79dee5cf3cf39feed761160e4797"} Aug 05 17:21:33.687047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:33.687009 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2m4bd" event={"ID":"cd322441-fd87-4da5-babe-874dda982d83","Type":"ContainerStarted","Data":"bc1291627a0a9c6c24d65dbf18201576f484ec09fabcd1d66f0b809704cc8cc1"} Aug 05 17:21:33.687461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:33.687088 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2m4bd" event={"ID":"cd322441-fd87-4da5-babe-874dda982d83","Type":"ContainerStarted","Data":"0babc6148dc74196b785673e8711d6f3a38c00cef6ce3e92a4ea477000e0e47a"} Aug 05 17:21:33.704959 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:33.704890 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2m4bd" podStartSLOduration=128.814769205 podStartE2EDuration="2m9.704869082s" totalImagesPullingTime="890.099877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:33.70202299 +0000 UTC m=+131.103251805" watchObservedRunningTime="2026-08-05 17:21:33.704869082 +0000 UTC m=+131.106097898" Aug 05 17:21:36.529837 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.529797 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" podUID="72739ecf-e267-4cc8-b2f3-be3d56767b3b" containerName="registry" containerID="cri-o://cffb71e3d1b2eb2392d5e5f1b264e8f8755cfa2f0911520ad3b011614b520edd" gracePeriod=30 Aug 05 17:21:36.700323 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.700292 2575 generic.go:356] "Generic (PLEG): container finished" podID="72739ecf-e267-4cc8-b2f3-be3d56767b3b" containerID="cffb71e3d1b2eb2392d5e5f1b264e8f8755cfa2f0911520ad3b011614b520edd" exitCode=0 Aug 05 17:21:36.700476 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.700368 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" event={"ID":"72739ecf-e267-4cc8-b2f3-be3d56767b3b","Type":"ContainerDied","Data":"cffb71e3d1b2eb2392d5e5f1b264e8f8755cfa2f0911520ad3b011614b520edd"} Aug 05 17:21:36.778468 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.778444 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:36.862491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862451 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-866w4\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-kube-api-access-866w4\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862489 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-bound-sa-token\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862704 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862520 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862704 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862537 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-certificates\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862704 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862688 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72739ecf-e267-4cc8-b2f3-be3d56767b3b-ca-trust-extracted\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862858 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862841 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-installation-pull-secrets\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862918 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862876 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-image-registry-private-configuration\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862976 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862917 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-trusted-ca\") pod \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\" (UID: \"72739ecf-e267-4cc8-b2f3-be3d56767b3b\") " Aug 05 17:21:36.862976 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.862925 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:21:36.863293 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.863270 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-certificates\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.863785 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.863755 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:21:36.865150 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.865093 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:21:36.865305 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.865285 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-kube-api-access-866w4" (OuterVolumeSpecName: "kube-api-access-866w4") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "kube-api-access-866w4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:21:36.865366 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.865317 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:21:36.865366 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.865325 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:21:36.865468 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.865422 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:21:36.870554 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.870531 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72739ecf-e267-4cc8-b2f3-be3d56767b3b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "72739ecf-e267-4cc8-b2f3-be3d56767b3b" (UID: "72739ecf-e267-4cc8-b2f3-be3d56767b3b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:21:36.964695 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964658 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72739ecf-e267-4cc8-b2f3-be3d56767b3b-trusted-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.964695 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964689 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-866w4\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-kube-api-access-866w4\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.964695 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964700 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-bound-sa-token\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.964922 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964708 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/72739ecf-e267-4cc8-b2f3-be3d56767b3b-registry-tls\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.964922 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964717 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/72739ecf-e267-4cc8-b2f3-be3d56767b3b-ca-trust-extracted\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.964922 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964724 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-installation-pull-secrets\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:36.964922 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:36.964736 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/72739ecf-e267-4cc8-b2f3-be3d56767b3b-image-registry-private-configuration\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:21:37.705896 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:37.705870 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" Aug 05 17:21:37.706310 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:37.705867 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765d5c5557-4zdr8" event={"ID":"72739ecf-e267-4cc8-b2f3-be3d56767b3b","Type":"ContainerDied","Data":"bb794ede3415085cdfa20afa829ae4f28a9835c3bfd8b3e96322cb1c88845574"} Aug 05 17:21:37.706310 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:37.705993 2575 scope.go:117] "RemoveContainer" containerID="cffb71e3d1b2eb2392d5e5f1b264e8f8755cfa2f0911520ad3b011614b520edd" Aug 05 17:21:37.721676 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:37.721648 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-765d5c5557-4zdr8"] Aug 05 17:21:37.725162 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:37.725142 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-765d5c5557-4zdr8"] Aug 05 17:21:39.025111 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:39.025073 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72739ecf-e267-4cc8-b2f3-be3d56767b3b" path="/var/lib/kubelet/pods/72739ecf-e267-4cc8-b2f3-be3d56767b3b/volumes" Aug 05 17:21:39.461787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:39.461746 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:39.461960 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:39.461817 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:39.466579 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:39.466558 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:39.720373 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:39.720300 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:21:39.770873 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:39.770831 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-766d686f4b-jh5fz"] Aug 05 17:21:41.725823 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:41.725787 2575 generic.go:356] "Generic (PLEG): container finished" podID="fe5d030a-f113-49df-b653-5673455bf156" containerID="07575b7210665f0aff31332984a3b612220990f110ab79559cf9c74fad70ada2" exitCode=0 Aug 05 17:21:41.726301 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:41.725868 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" event={"ID":"fe5d030a-f113-49df-b653-5673455bf156","Type":"ContainerDied","Data":"07575b7210665f0aff31332984a3b612220990f110ab79559cf9c74fad70ada2"} Aug 05 17:21:41.726301 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:41.726215 2575 scope.go:117] "RemoveContainer" containerID="07575b7210665f0aff31332984a3b612220990f110ab79559cf9c74fad70ada2" Aug 05 17:21:41.727531 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:41.727510 2575 generic.go:356] "Generic (PLEG): container finished" podID="36009ed7-f9b0-48d3-920e-3d6d452d18e8" containerID="72218df46390f72b6d5b42eab65b9b63572ba875b5763559bc2b231f58eaa3d8" exitCode=0 Aug 05 17:21:41.727612 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:41.727560 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-75ch2" event={"ID":"36009ed7-f9b0-48d3-920e-3d6d452d18e8","Type":"ContainerDied","Data":"72218df46390f72b6d5b42eab65b9b63572ba875b5763559bc2b231f58eaa3d8"} Aug 05 17:21:41.727816 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:41.727804 2575 scope.go:117] "RemoveContainer" containerID="72218df46390f72b6d5b42eab65b9b63572ba875b5763559bc2b231f58eaa3d8" Aug 05 17:21:42.736650 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:42.736592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-b8lxd" event={"ID":"fe5d030a-f113-49df-b653-5673455bf156","Type":"ContainerStarted","Data":"b337928fb50d9ace5ecbed70d2d44b845d6701bcb71ea06d35bd8589787ddcbe"} Aug 05 17:21:42.738479 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:42.738447 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-75ch2" event={"ID":"36009ed7-f9b0-48d3-920e-3d6d452d18e8","Type":"ContainerStarted","Data":"e313cc375941e71fc4115da682ae42a2ba973016a137d67dd01a1214bf2ad84c"} Aug 05 17:21:42.808542 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:42.807798 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p8tz8_298eeb1d-3ed1-407f-990f-08b90ccd84f5/serve-healthcheck-canary/0.log" Aug 05 17:21:49.029226 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:49.029194 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:21:49.032869 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:21:49.032848 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-d54cd6548-xbhpt" Aug 05 17:22:04.790514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:04.790447 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-766d686f4b-jh5fz" podUID="6299984f-379e-4258-afae-886df0235139" containerName="console" containerID="cri-o://7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222" gracePeriod=15 Aug 05 17:22:05.033321 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.033295 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:22:05.113331 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113303 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-oauth-serving-cert\") pod \"6299984f-379e-4258-afae-886df0235139\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " Aug 05 17:22:05.113491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113347 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkl9h\" (UniqueName: \"kubernetes.io/projected/6299984f-379e-4258-afae-886df0235139-kube-api-access-rkl9h\") pod \"6299984f-379e-4258-afae-886df0235139\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " Aug 05 17:22:05.113491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113379 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-serving-cert\") pod \"6299984f-379e-4258-afae-886df0235139\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " Aug 05 17:22:05.113491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113422 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-console-config\") pod \"6299984f-379e-4258-afae-886df0235139\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " Aug 05 17:22:05.113491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113451 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-oauth-config\") pod \"6299984f-379e-4258-afae-886df0235139\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " Aug 05 17:22:05.113670 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113532 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-service-ca\") pod \"6299984f-379e-4258-afae-886df0235139\" (UID: \"6299984f-379e-4258-afae-886df0235139\") " Aug 05 17:22:05.113769 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113736 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6299984f-379e-4258-afae-886df0235139" (UID: "6299984f-379e-4258-afae-886df0235139"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:22:05.113987 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113897 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-console-config" (OuterVolumeSpecName: "console-config") pod "6299984f-379e-4258-afae-886df0235139" (UID: "6299984f-379e-4258-afae-886df0235139"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:22:05.114097 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.113966 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-service-ca" (OuterVolumeSpecName: "service-ca") pod "6299984f-379e-4258-afae-886df0235139" (UID: "6299984f-379e-4258-afae-886df0235139"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:22:05.115739 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.115715 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6299984f-379e-4258-afae-886df0235139" (UID: "6299984f-379e-4258-afae-886df0235139"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:22:05.115826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.115748 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6299984f-379e-4258-afae-886df0235139" (UID: "6299984f-379e-4258-afae-886df0235139"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:22:05.115826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.115760 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6299984f-379e-4258-afae-886df0235139-kube-api-access-rkl9h" (OuterVolumeSpecName: "kube-api-access-rkl9h") pod "6299984f-379e-4258-afae-886df0235139" (UID: "6299984f-379e-4258-afae-886df0235139"). InnerVolumeSpecName "kube-api-access-rkl9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:22:05.214503 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.214464 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:22:05.214503 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.214494 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-oauth-serving-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:22:05.214503 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.214505 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkl9h\" (UniqueName: \"kubernetes.io/projected/6299984f-379e-4258-afae-886df0235139-kube-api-access-rkl9h\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:22:05.214503 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.214515 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-serving-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:22:05.214740 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.214524 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6299984f-379e-4258-afae-886df0235139-console-config\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:22:05.214740 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.214533 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6299984f-379e-4258-afae-886df0235139-console-oauth-config\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:22:05.840730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.840695 2575 generic.go:356] "Generic (PLEG): container finished" podID="6299984f-379e-4258-afae-886df0235139" containerID="7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222" exitCode=0 Aug 05 17:22:05.841228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.840742 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766d686f4b-jh5fz" Aug 05 17:22:05.841228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.840779 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766d686f4b-jh5fz" event={"ID":"6299984f-379e-4258-afae-886df0235139","Type":"ContainerDied","Data":"7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222"} Aug 05 17:22:05.841228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.840832 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766d686f4b-jh5fz" event={"ID":"6299984f-379e-4258-afae-886df0235139","Type":"ContainerDied","Data":"9759231a7ca2766309a557fb84740a7b9e12c1b58a392e64d0c63fd3f0b323ce"} Aug 05 17:22:05.841228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.840852 2575 scope.go:117] "RemoveContainer" containerID="7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222" Aug 05 17:22:05.849598 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.849581 2575 scope.go:117] "RemoveContainer" containerID="7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222" Aug 05 17:22:05.849840 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:22:05.849816 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222\": container with ID starting with 7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222 not found: ID does not exist" containerID="7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222" Aug 05 17:22:05.849933 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.849848 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222"} err="failed to get container status \"7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222\": rpc error: code = NotFound desc = could not find container \"7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222\": container with ID starting with 7a494f1bddbf393f9f5425df5df410c59e79dd71560486a181dde32e68bf7222 not found: ID does not exist" Aug 05 17:22:05.862782 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.862756 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-766d686f4b-jh5fz"] Aug 05 17:22:05.866811 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:05.866782 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-766d686f4b-jh5fz"] Aug 05 17:22:07.024995 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:07.024961 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6299984f-379e-4258-afae-886df0235139" path="/var/lib/kubelet/pods/6299984f-379e-4258-afae-886df0235139/volumes" Aug 05 17:22:26.866593 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.866559 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bddf6b9f7-nbxrr"] Aug 05 17:22:26.867105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.866936 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72739ecf-e267-4cc8-b2f3-be3d56767b3b" containerName="registry" Aug 05 17:22:26.867105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.866951 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="72739ecf-e267-4cc8-b2f3-be3d56767b3b" containerName="registry" Aug 05 17:22:26.867105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.866966 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6299984f-379e-4258-afae-886df0235139" containerName="console" Aug 05 17:22:26.867105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.866972 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6299984f-379e-4258-afae-886df0235139" containerName="console" Aug 05 17:22:26.867105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.867040 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="72739ecf-e267-4cc8-b2f3-be3d56767b3b" containerName="registry" Aug 05 17:22:26.867105 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.867067 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6299984f-379e-4258-afae-886df0235139" containerName="console" Aug 05 17:22:26.871401 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.871377 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.879132 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.879109 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bddf6b9f7-nbxrr"] Aug 05 17:22:26.887458 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887432 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-oauth-config\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.887574 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887494 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4f9\" (UniqueName: \"kubernetes.io/projected/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-kube-api-access-lz4f9\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.887574 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-trusted-ca-bundle\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.887574 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-serving-cert\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.887679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887587 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-oauth-serving-cert\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.887679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887605 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-service-ca\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.887679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.887622 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-config\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988416 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988376 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-trusted-ca-bundle\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-serving-cert\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988474 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-oauth-serving-cert\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988498 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-service-ca\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-config\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-oauth-config\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.988836 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.988637 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4f9\" (UniqueName: \"kubernetes.io/projected/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-kube-api-access-lz4f9\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.989387 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.989362 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-oauth-serving-cert\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.989387 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.989370 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-service-ca\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.989539 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.989398 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-trusted-ca-bundle\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.989539 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.989419 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-config\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.991488 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.991469 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-oauth-config\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:26.991594 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:26.991578 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-serving-cert\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:27.005335 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.005308 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4f9\" (UniqueName: \"kubernetes.io/projected/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-kube-api-access-lz4f9\") pod \"console-bddf6b9f7-nbxrr\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:27.028823 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.028793 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 17:22:27.033691 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.033673 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.036047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036027 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 05 17:22:27.036196 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036172 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 05 17:22:27.036263 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036218 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rdkj5\"" Aug 05 17:22:27.036263 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036240 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 05 17:22:27.036263 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036251 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 05 17:22:27.036399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036261 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 05 17:22:27.036455 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036441 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 05 17:22:27.036690 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036661 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 05 17:22:27.036774 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.036758 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 05 17:22:27.042849 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.042830 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 05 17:22:27.053609 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.053588 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 17:22:27.089879 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.089853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cdjd\" (UniqueName: \"kubernetes.io/projected/c7145719-1731-469b-af9d-9d8decda5652-kube-api-access-7cdjd\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090030 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.089885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090030 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.089911 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7145719-1731-469b-af9d-9d8decda5652-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090030 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.089962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7145719-1731-469b-af9d-9d8decda5652-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090030 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.089992 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7145719-1731-469b-af9d-9d8decda5652-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090277 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090277 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090079 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-web-config\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090277 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090100 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090277 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c7145719-1731-469b-af9d-9d8decda5652-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090277 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090178 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-config-volume\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090277 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090207 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090307 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7145719-1731-469b-af9d-9d8decda5652-config-out\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.090491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.090340 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.180742 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.180655 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:27.191386 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191359 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7145719-1731-469b-af9d-9d8decda5652-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191493 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7145719-1731-469b-af9d-9d8decda5652-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191493 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191430 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7145719-1731-469b-af9d-9d8decda5652-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191493 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191642 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-web-config\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191642 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191642 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c7145719-1731-469b-af9d-9d8decda5652-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191642 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191576 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-config-volume\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191642 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191603 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7145719-1731-469b-af9d-9d8decda5652-config-out\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191698 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cdjd\" (UniqueName: \"kubernetes.io/projected/c7145719-1731-469b-af9d-9d8decda5652-kube-api-access-7cdjd\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.191855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.191769 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.192181 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.192150 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c7145719-1731-469b-af9d-9d8decda5652-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.192867 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.192555 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7145719-1731-469b-af9d-9d8decda5652-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.193588 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.193550 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7145719-1731-469b-af9d-9d8decda5652-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.194927 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.194904 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.196353 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.196329 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-config-volume\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.196997 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.196779 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c7145719-1731-469b-af9d-9d8decda5652-config-out\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.196997 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.196923 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.197183 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.197004 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c7145719-1731-469b-af9d-9d8decda5652-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.197265 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.197240 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.197265 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.197258 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.197400 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.197380 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.197688 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.197666 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c7145719-1731-469b-af9d-9d8decda5652-web-config\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.203299 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.203278 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cdjd\" (UniqueName: \"kubernetes.io/projected/c7145719-1731-469b-af9d-9d8decda5652-kube-api-access-7cdjd\") pod \"alertmanager-main-0\" (UID: \"c7145719-1731-469b-af9d-9d8decda5652\") " pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.309538 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.309502 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bddf6b9f7-nbxrr"] Aug 05 17:22:27.312168 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:22:27.312142 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bc1eba_0c20_4f6b_b3bd_e0e335dce068.slice/crio-6d547221471c969d0a75432817528bc078e1a91cdd03ca22064485c9aa16f8b8 WatchSource:0}: Error finding container 6d547221471c969d0a75432817528bc078e1a91cdd03ca22064485c9aa16f8b8: Status 404 returned error can't find the container with id 6d547221471c969d0a75432817528bc078e1a91cdd03ca22064485c9aa16f8b8 Aug 05 17:22:27.343859 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.343824 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 05 17:22:27.488374 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.488345 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 05 17:22:27.491367 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:22:27.491340 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7145719_1731_469b_af9d_9d8decda5652.slice/crio-fe63d9ce3dd40d3fb9c849b53542395ee07cb1e7dbd93219ba2f7c7abc236240 WatchSource:0}: Error finding container fe63d9ce3dd40d3fb9c849b53542395ee07cb1e7dbd93219ba2f7c7abc236240: Status 404 returned error can't find the container with id fe63d9ce3dd40d3fb9c849b53542395ee07cb1e7dbd93219ba2f7c7abc236240 Aug 05 17:22:27.929960 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.929926 2575 generic.go:356] "Generic (PLEG): container finished" podID="c7145719-1731-469b-af9d-9d8decda5652" containerID="f33c708cd8c71e089ce12ac705281d84f4d0e5509279da70fa2b63b32600f53f" exitCode=0 Aug 05 17:22:27.930436 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.930003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerDied","Data":"f33c708cd8c71e089ce12ac705281d84f4d0e5509279da70fa2b63b32600f53f"} Aug 05 17:22:27.930436 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.930108 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"fe63d9ce3dd40d3fb9c849b53542395ee07cb1e7dbd93219ba2f7c7abc236240"} Aug 05 17:22:27.931597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.931536 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bddf6b9f7-nbxrr" event={"ID":"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068","Type":"ContainerStarted","Data":"d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835"} Aug 05 17:22:27.931597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.931576 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bddf6b9f7-nbxrr" event={"ID":"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068","Type":"ContainerStarted","Data":"6d547221471c969d0a75432817528bc078e1a91cdd03ca22064485c9aa16f8b8"} Aug 05 17:22:27.981474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:27.981413 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bddf6b9f7-nbxrr" podStartSLOduration=1.9813959639999998 podStartE2EDuration="1.981395964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:27.980889993 +0000 UTC m=+185.382118807" watchObservedRunningTime="2026-08-05 17:22:27.981395964 +0000 UTC m=+185.382624778" Aug 05 17:22:29.943268 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.943226 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"7d081d3ee2d1a63106a924bfe564e3ba6259c92bbdb5b1954e9b25392a6073ee"} Aug 05 17:22:29.943699 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.943323 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"32de8607e7112495411cec9305efa180d5f47ce1481ce1614ac0e884789d6804"} Aug 05 17:22:29.943699 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.943369 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"3b13d61dc5d79ad69e482db8e6dac957f00132a90efefa65bb5f00486276a2fd"} Aug 05 17:22:29.943699 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.943415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"0a73a6563ea3b8abeece3436999cab96f04894ba0ba40f2cd5e08d831298e4f6"} Aug 05 17:22:29.943699 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.943451 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"028ec4381f63e56c680bb40a03a1018b1a8c3c3dafb10a9b5ebbc3b213b364af"} Aug 05 17:22:29.943699 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.943489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c7145719-1731-469b-af9d-9d8decda5652","Type":"ContainerStarted","Data":"42953271b4aa8d209e51aaaf5caa30f53e0dd53b3e7e01479b8cdef8cf4a973e"} Aug 05 17:22:29.969921 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:29.969847 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.631633389 podStartE2EDuration="3.969833557s" totalImagesPullingTime="1.323385502s" totalInitContainerRuntime="14.814666ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:29.968536855 +0000 UTC m=+187.369765670" watchObservedRunningTime="2026-08-05 17:22:29.969833557 +0000 UTC m=+187.371062400" Aug 05 17:22:37.181252 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:37.181202 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:37.181665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:37.181297 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:37.186128 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:37.186106 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:37.978270 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:37.978242 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:22:38.031439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:22:38.031409 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797c79c75b-7qsjx"] Aug 05 17:23:03.051017 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.050965 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-797c79c75b-7qsjx" podUID="fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" containerName="console" containerID="cri-o://4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee" gracePeriod=15 Aug 05 17:23:03.287070 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.287035 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:23:03.397306 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397223 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m2jl\" (UniqueName: \"kubernetes.io/projected/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-kube-api-access-9m2jl\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397306 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397265 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-serving-cert\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397306 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397282 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-service-ca\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397576 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397326 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-trusted-ca-bundle\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397576 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397346 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-config\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397576 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397401 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-oauth-config\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397576 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397448 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-oauth-serving-cert\") pod \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\" (UID: \"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3\") " Aug 05 17:23:03.397779 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397685 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:23:03.397858 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397828 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:23:03.397931 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397797 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-config" (OuterVolumeSpecName: "console-config") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:23:03.397986 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.397949 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:23:03.399421 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.399389 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:23:03.399581 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.399555 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-kube-api-access-9m2jl" (OuterVolumeSpecName: "kube-api-access-9m2jl") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "kube-api-access-9m2jl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:23:03.399622 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.399578 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" (UID: "fd814358-3df6-4d1f-bcf7-8efb6a1db3f3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:23:03.498043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498008 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-trusted-ca-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:03.498043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498032 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-config\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:03.498043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498043 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-oauth-config\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:03.498043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498062 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-oauth-serving-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:03.498402 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498071 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9m2jl\" (UniqueName: \"kubernetes.io/projected/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-kube-api-access-9m2jl\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:03.498402 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498079 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-console-serving-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:03.498402 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:03.498086 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:23:04.079643 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.079607 2575 generic.go:356] "Generic (PLEG): container finished" podID="fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" containerID="4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee" exitCode=0 Aug 05 17:23:04.080137 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.079653 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c79c75b-7qsjx" Aug 05 17:23:04.080137 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.079669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c79c75b-7qsjx" event={"ID":"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3","Type":"ContainerDied","Data":"4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee"} Aug 05 17:23:04.080137 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.079710 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c79c75b-7qsjx" event={"ID":"fd814358-3df6-4d1f-bcf7-8efb6a1db3f3","Type":"ContainerDied","Data":"b7f3004da910e486ef1a939d706e663373e9ceaeb8e2b7291df392ed5034465e"} Aug 05 17:23:04.080137 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.079729 2575 scope.go:117] "RemoveContainer" containerID="4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee" Aug 05 17:23:04.088494 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.088476 2575 scope.go:117] "RemoveContainer" containerID="4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee" Aug 05 17:23:04.088724 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:23:04.088708 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee\": container with ID starting with 4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee not found: ID does not exist" containerID="4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee" Aug 05 17:23:04.088760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.088733 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee"} err="failed to get container status \"4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee\": rpc error: code = NotFound desc = could not find container \"4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee\": container with ID starting with 4958d3ae7874def7341ea2de74adc66b7a7abee5f56c43afaa7176ea9120d2ee not found: ID does not exist" Aug 05 17:23:04.101030 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.101004 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797c79c75b-7qsjx"] Aug 05 17:23:04.105500 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:04.105478 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-797c79c75b-7qsjx"] Aug 05 17:23:05.024996 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:23:05.024963 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" path="/var/lib/kubelet/pods/fd814358-3df6-4d1f-bcf7-8efb6a1db3f3/volumes" Aug 05 17:24:21.134985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.134945 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5"] Aug 05 17:24:21.135530 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.135450 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" containerName="console" Aug 05 17:24:21.135530 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.135470 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" containerName="console" Aug 05 17:24:21.135653 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.135577 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd814358-3df6-4d1f-bcf7-8efb6a1db3f3" containerName="console" Aug 05 17:24:21.138766 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.138744 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.141046 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.141028 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 05 17:24:21.141174 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.141152 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 05 17:24:21.141660 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.141647 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lw45v\"" Aug 05 17:24:21.147073 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.147035 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5"] Aug 05 17:24:21.162478 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.162456 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krb7w\" (UniqueName: \"kubernetes.io/projected/8d774b7c-a914-4dfe-bad7-89defe366644-kube-api-access-krb7w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.162570 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.162502 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.162570 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.162529 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.263237 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.263207 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krb7w\" (UniqueName: \"kubernetes.io/projected/8d774b7c-a914-4dfe-bad7-89defe366644-kube-api-access-krb7w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.263396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.263280 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.263396 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.263317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.263629 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.263610 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.263701 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.263648 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.271249 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.271228 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krb7w\" (UniqueName: \"kubernetes.io/projected/8d774b7c-a914-4dfe-bad7-89defe366644-kube-api-access-krb7w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.447714 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.447646 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:21.569482 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:21.569455 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5"] Aug 05 17:24:21.572036 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:24:21.572010 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d774b7c_a914_4dfe_bad7_89defe366644.slice/crio-6967cc9d04deb319e1c38d9393dd253359cbe54ef0f5c2b5012d6ade4bb20337 WatchSource:0}: Error finding container 6967cc9d04deb319e1c38d9393dd253359cbe54ef0f5c2b5012d6ade4bb20337: Status 404 returned error can't find the container with id 6967cc9d04deb319e1c38d9393dd253359cbe54ef0f5c2b5012d6ade4bb20337 Aug 05 17:24:22.412073 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:22.412019 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" event={"ID":"8d774b7c-a914-4dfe-bad7-89defe366644","Type":"ContainerStarted","Data":"6967cc9d04deb319e1c38d9393dd253359cbe54ef0f5c2b5012d6ade4bb20337"} Aug 05 17:24:22.942669 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:22.942640 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:24:22.946726 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:22.946701 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:24:22.950329 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:22.950309 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:24:22.954596 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:22.954572 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:24:22.954954 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:22.954939 2575 kubelet.go:1648] "Image garbage collection succeeded" Aug 05 17:24:27.435278 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:27.435241 2575 generic.go:356] "Generic (PLEG): container finished" podID="8d774b7c-a914-4dfe-bad7-89defe366644" containerID="f928468b53429578b3de72a27dffe1a6dec3d81ef5305ddfb8d22c632049ed62" exitCode=0 Aug 05 17:24:27.435278 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:27.435278 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" event={"ID":"8d774b7c-a914-4dfe-bad7-89defe366644","Type":"ContainerDied","Data":"f928468b53429578b3de72a27dffe1a6dec3d81ef5305ddfb8d22c632049ed62"} Aug 05 17:24:27.436420 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:27.436404 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 17:24:30.450489 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:30.450453 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" event={"ID":"8d774b7c-a914-4dfe-bad7-89defe366644","Type":"ContainerStarted","Data":"49c8f9900ac12d5b0d9b03a350dfd6b4223eeff687c67f5208cc9a873882b63a"} Aug 05 17:24:31.456395 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:31.456360 2575 generic.go:356] "Generic (PLEG): container finished" podID="8d774b7c-a914-4dfe-bad7-89defe366644" containerID="49c8f9900ac12d5b0d9b03a350dfd6b4223eeff687c67f5208cc9a873882b63a" exitCode=0 Aug 05 17:24:31.456877 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:31.456403 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" event={"ID":"8d774b7c-a914-4dfe-bad7-89defe366644","Type":"ContainerDied","Data":"49c8f9900ac12d5b0d9b03a350dfd6b4223eeff687c67f5208cc9a873882b63a"} Aug 05 17:24:38.490131 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:38.490089 2575 generic.go:356] "Generic (PLEG): container finished" podID="8d774b7c-a914-4dfe-bad7-89defe366644" containerID="596f1154f9ed103c869801dda2d383fee4e2cdf03cc9bea082c220748316dba3" exitCode=0 Aug 05 17:24:38.490518 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:38.490176 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" event={"ID":"8d774b7c-a914-4dfe-bad7-89defe366644","Type":"ContainerDied","Data":"596f1154f9ed103c869801dda2d383fee4e2cdf03cc9bea082c220748316dba3"} Aug 05 17:24:39.618029 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.618005 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:39.734267 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.734236 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krb7w\" (UniqueName: \"kubernetes.io/projected/8d774b7c-a914-4dfe-bad7-89defe366644-kube-api-access-krb7w\") pod \"8d774b7c-a914-4dfe-bad7-89defe366644\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " Aug 05 17:24:39.734267 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.734270 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-util\") pod \"8d774b7c-a914-4dfe-bad7-89defe366644\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " Aug 05 17:24:39.734485 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.734466 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-bundle\") pod \"8d774b7c-a914-4dfe-bad7-89defe366644\" (UID: \"8d774b7c-a914-4dfe-bad7-89defe366644\") " Aug 05 17:24:39.735079 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.735033 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-bundle" (OuterVolumeSpecName: "bundle") pod "8d774b7c-a914-4dfe-bad7-89defe366644" (UID: "8d774b7c-a914-4dfe-bad7-89defe366644"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:24:39.736686 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.736659 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d774b7c-a914-4dfe-bad7-89defe366644-kube-api-access-krb7w" (OuterVolumeSpecName: "kube-api-access-krb7w") pod "8d774b7c-a914-4dfe-bad7-89defe366644" (UID: "8d774b7c-a914-4dfe-bad7-89defe366644"). InnerVolumeSpecName "kube-api-access-krb7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:24:39.738326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.738306 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-util" (OuterVolumeSpecName: "util") pod "8d774b7c-a914-4dfe-bad7-89defe366644" (UID: "8d774b7c-a914-4dfe-bad7-89defe366644"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:24:39.835230 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.835197 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:24:39.835230 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.835229 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krb7w\" (UniqueName: \"kubernetes.io/projected/8d774b7c-a914-4dfe-bad7-89defe366644-kube-api-access-krb7w\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:24:39.835230 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:39.835239 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d774b7c-a914-4dfe-bad7-89defe366644-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:24:40.500358 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:40.500329 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" Aug 05 17:24:40.500551 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:40.500329 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754059xk5" event={"ID":"8d774b7c-a914-4dfe-bad7-89defe366644","Type":"ContainerDied","Data":"6967cc9d04deb319e1c38d9393dd253359cbe54ef0f5c2b5012d6ade4bb20337"} Aug 05 17:24:40.500551 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:40.500446 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6967cc9d04deb319e1c38d9393dd253359cbe54ef0f5c2b5012d6ade4bb20337" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.124184 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj"] Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.124939 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="extract" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.124959 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="extract" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.124983 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="util" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.124990 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="util" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.124999 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="pull" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.125006 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="pull" Aug 05 17:24:44.126832 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.125176 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d774b7c-a914-4dfe-bad7-89defe366644" containerName="extract" Aug 05 17:24:44.165821 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.165786 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj"] Aug 05 17:24:44.165968 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.165874 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.168619 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.168591 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Aug 05 17:24:44.168619 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.168614 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zprhb\"" Aug 05 17:24:44.168823 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.168808 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:24:44.270482 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.270447 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aca831d8-4561-49aa-938b-086e25a0b186-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pj2dj\" (UID: \"aca831d8-4561-49aa-938b-086e25a0b186\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.270644 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.270515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kplp6\" (UniqueName: \"kubernetes.io/projected/aca831d8-4561-49aa-938b-086e25a0b186-kube-api-access-kplp6\") pod \"cert-manager-operator-controller-manager-55c444bf86-pj2dj\" (UID: \"aca831d8-4561-49aa-938b-086e25a0b186\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.371068 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.371015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aca831d8-4561-49aa-938b-086e25a0b186-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pj2dj\" (UID: \"aca831d8-4561-49aa-938b-086e25a0b186\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.371220 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.371113 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kplp6\" (UniqueName: \"kubernetes.io/projected/aca831d8-4561-49aa-938b-086e25a0b186-kube-api-access-kplp6\") pod \"cert-manager-operator-controller-manager-55c444bf86-pj2dj\" (UID: \"aca831d8-4561-49aa-938b-086e25a0b186\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.371404 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.371386 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aca831d8-4561-49aa-938b-086e25a0b186-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-pj2dj\" (UID: \"aca831d8-4561-49aa-938b-086e25a0b186\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.379074 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.379009 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kplp6\" (UniqueName: \"kubernetes.io/projected/aca831d8-4561-49aa-938b-086e25a0b186-kube-api-access-kplp6\") pod \"cert-manager-operator-controller-manager-55c444bf86-pj2dj\" (UID: \"aca831d8-4561-49aa-938b-086e25a0b186\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.475534 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.475498 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" Aug 05 17:24:44.619549 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:44.619517 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj"] Aug 05 17:24:44.623302 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:24:44.623274 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca831d8_4561_49aa_938b_086e25a0b186.slice/crio-f5c054ed0627789aa1b2589ba4bef674b007f6bba7c7bac9b2748c39a6ea4566 WatchSource:0}: Error finding container f5c054ed0627789aa1b2589ba4bef674b007f6bba7c7bac9b2748c39a6ea4566: Status 404 returned error can't find the container with id f5c054ed0627789aa1b2589ba4bef674b007f6bba7c7bac9b2748c39a6ea4566 Aug 05 17:24:45.526730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:45.526690 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" event={"ID":"aca831d8-4561-49aa-938b-086e25a0b186","Type":"ContainerStarted","Data":"f5c054ed0627789aa1b2589ba4bef674b007f6bba7c7bac9b2748c39a6ea4566"} Aug 05 17:24:47.537068 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:47.537015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" event={"ID":"aca831d8-4561-49aa-938b-086e25a0b186","Type":"ContainerStarted","Data":"e5b12824a94fd56e79e49cc8b374b2862153abf7d14d05c6a8e59dafa8f64ebb"} Aug 05 17:24:47.559027 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:47.558981 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-pj2dj" podStartSLOduration=1.364143678 podStartE2EDuration="3.558962934s" totalImagesPullingTime="2.194819256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:47.557354884 +0000 UTC m=+324.958583714" watchObservedRunningTime="2026-08-05 17:24:47.558962934 +0000 UTC m=+324.960191747" Aug 05 17:24:48.565303 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.565274 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8"] Aug 05 17:24:48.579261 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.579237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.582224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.582201 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 05 17:24:48.582421 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.582408 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lw45v\"" Aug 05 17:24:48.583316 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.583297 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 05 17:24:48.587546 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.587519 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8"] Aug 05 17:24:48.709742 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.709700 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.709742 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.709742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhl8s\" (UniqueName: \"kubernetes.io/projected/0c8abd1b-3c5d-410e-a239-edffa7f41984-kube-api-access-bhl8s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.710008 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.709844 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.810350 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.810318 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.810496 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.810407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.810496 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.810440 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhl8s\" (UniqueName: \"kubernetes.io/projected/0c8abd1b-3c5d-410e-a239-edffa7f41984-kube-api-access-bhl8s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.810758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.810736 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.810809 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.810758 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.819500 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.819441 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhl8s\" (UniqueName: \"kubernetes.io/projected/0c8abd1b-3c5d-410e-a239-edffa7f41984-kube-api-access-bhl8s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:48.888835 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:48.888799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:49.049732 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:49.049652 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8"] Aug 05 17:24:49.052424 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:24:49.052393 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8abd1b_3c5d_410e_a239_edffa7f41984.slice/crio-86fe6e1b67dfd6020bd23f47be158fcc9160a480fb70bb423a4a5aa89f842d2f WatchSource:0}: Error finding container 86fe6e1b67dfd6020bd23f47be158fcc9160a480fb70bb423a4a5aa89f842d2f: Status 404 returned error can't find the container with id 86fe6e1b67dfd6020bd23f47be158fcc9160a480fb70bb423a4a5aa89f842d2f Aug 05 17:24:49.547208 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:49.547122 2575 generic.go:356] "Generic (PLEG): container finished" podID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerID="d0d4c28d60be25a08740b92237b8b8054c6398da3fbc7693ee1df248fecd5775" exitCode=0 Aug 05 17:24:49.547208 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:49.547195 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" event={"ID":"0c8abd1b-3c5d-410e-a239-edffa7f41984","Type":"ContainerDied","Data":"d0d4c28d60be25a08740b92237b8b8054c6398da3fbc7693ee1df248fecd5775"} Aug 05 17:24:49.547390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:49.547234 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" event={"ID":"0c8abd1b-3c5d-410e-a239-edffa7f41984","Type":"ContainerStarted","Data":"86fe6e1b67dfd6020bd23f47be158fcc9160a480fb70bb423a4a5aa89f842d2f"} Aug 05 17:24:50.120617 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.120541 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z"] Aug 05 17:24:50.141372 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.141341 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z"] Aug 05 17:24:50.141523 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.141451 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.143707 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.143685 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 05 17:24:50.144573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.144559 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lltnd\"" Aug 05 17:24:50.144573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.144577 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 05 17:24:50.325432 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.325378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbhz2\" (UniqueName: \"kubernetes.io/projected/a469f52b-4194-41b1-8e9f-505860fabc31-kube-api-access-jbhz2\") pod \"cert-manager-cainjector-559cdd8c87-nkv8z\" (UID: \"a469f52b-4194-41b1-8e9f-505860fabc31\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.325607 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.325439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a469f52b-4194-41b1-8e9f-505860fabc31-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nkv8z\" (UID: \"a469f52b-4194-41b1-8e9f-505860fabc31\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.426021 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.425941 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbhz2\" (UniqueName: \"kubernetes.io/projected/a469f52b-4194-41b1-8e9f-505860fabc31-kube-api-access-jbhz2\") pod \"cert-manager-cainjector-559cdd8c87-nkv8z\" (UID: \"a469f52b-4194-41b1-8e9f-505860fabc31\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.426021 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.425985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a469f52b-4194-41b1-8e9f-505860fabc31-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nkv8z\" (UID: \"a469f52b-4194-41b1-8e9f-505860fabc31\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.435245 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.435217 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a469f52b-4194-41b1-8e9f-505860fabc31-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-nkv8z\" (UID: \"a469f52b-4194-41b1-8e9f-505860fabc31\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.435376 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.435354 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbhz2\" (UniqueName: \"kubernetes.io/projected/a469f52b-4194-41b1-8e9f-505860fabc31-kube-api-access-jbhz2\") pod \"cert-manager-cainjector-559cdd8c87-nkv8z\" (UID: \"a469f52b-4194-41b1-8e9f-505860fabc31\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.460346 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.460315 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" Aug 05 17:24:50.585237 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:50.585212 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z"] Aug 05 17:24:50.586918 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:24:50.586892 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda469f52b_4194_41b1_8e9f_505860fabc31.slice/crio-d582866c34939a644bf58890ba5f6426f7cf86523070ab55d1f535ca983dbb40 WatchSource:0}: Error finding container d582866c34939a644bf58890ba5f6426f7cf86523070ab55d1f535ca983dbb40: Status 404 returned error can't find the container with id d582866c34939a644bf58890ba5f6426f7cf86523070ab55d1f535ca983dbb40 Aug 05 17:24:51.560575 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:51.560540 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" event={"ID":"a469f52b-4194-41b1-8e9f-505860fabc31","Type":"ContainerStarted","Data":"d582866c34939a644bf58890ba5f6426f7cf86523070ab55d1f535ca983dbb40"} Aug 05 17:24:52.568627 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:52.568586 2575 generic.go:356] "Generic (PLEG): container finished" podID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerID="2aaefb6a81c447a76d6d12e9d02404c5cae44e8a54f05d3bb261e35f96611dc9" exitCode=0 Aug 05 17:24:52.569009 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:52.568674 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" event={"ID":"0c8abd1b-3c5d-410e-a239-edffa7f41984","Type":"ContainerDied","Data":"2aaefb6a81c447a76d6d12e9d02404c5cae44e8a54f05d3bb261e35f96611dc9"} Aug 05 17:24:53.575383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:53.575306 2575 generic.go:356] "Generic (PLEG): container finished" podID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerID="a642dd31feba044f2ce2ce2b2c580ce06826b89f813f4f8990e3b0d4f4cb44b1" exitCode=0 Aug 05 17:24:53.575778 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:53.575388 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" event={"ID":"0c8abd1b-3c5d-410e-a239-edffa7f41984","Type":"ContainerDied","Data":"a642dd31feba044f2ce2ce2b2c580ce06826b89f813f4f8990e3b0d4f4cb44b1"} Aug 05 17:24:53.576932 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:53.576907 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" event={"ID":"a469f52b-4194-41b1-8e9f-505860fabc31","Type":"ContainerStarted","Data":"c82e172ba160314d3dfeb8a6dcba69250ed29c55ef9e308239f59bedcdb447b8"} Aug 05 17:24:53.621984 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:53.621934 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-nkv8z" podStartSLOduration=0.852882892 podStartE2EDuration="3.621920978s" totalImagesPullingTime="2.769038086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:24:53.620866561 +0000 UTC m=+331.022095376" watchObservedRunningTime="2026-08-05 17:24:53.621920978 +0000 UTC m=+331.023149791" Aug 05 17:24:54.706912 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.706887 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:54.867992 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.867898 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhl8s\" (UniqueName: \"kubernetes.io/projected/0c8abd1b-3c5d-410e-a239-edffa7f41984-kube-api-access-bhl8s\") pod \"0c8abd1b-3c5d-410e-a239-edffa7f41984\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " Aug 05 17:24:54.868174 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.868038 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-util\") pod \"0c8abd1b-3c5d-410e-a239-edffa7f41984\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " Aug 05 17:24:54.868174 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.868083 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-bundle\") pod \"0c8abd1b-3c5d-410e-a239-edffa7f41984\" (UID: \"0c8abd1b-3c5d-410e-a239-edffa7f41984\") " Aug 05 17:24:54.868472 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.868442 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-bundle" (OuterVolumeSpecName: "bundle") pod "0c8abd1b-3c5d-410e-a239-edffa7f41984" (UID: "0c8abd1b-3c5d-410e-a239-edffa7f41984"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:24:54.869950 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.869924 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c8abd1b-3c5d-410e-a239-edffa7f41984-kube-api-access-bhl8s" (OuterVolumeSpecName: "kube-api-access-bhl8s") pod "0c8abd1b-3c5d-410e-a239-edffa7f41984" (UID: "0c8abd1b-3c5d-410e-a239-edffa7f41984"). InnerVolumeSpecName "kube-api-access-bhl8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:24:54.872884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.872861 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-util" (OuterVolumeSpecName: "util") pod "0c8abd1b-3c5d-410e-a239-edffa7f41984" (UID: "0c8abd1b-3c5d-410e-a239-edffa7f41984"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:24:54.969196 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.969166 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhl8s\" (UniqueName: \"kubernetes.io/projected/0c8abd1b-3c5d-410e-a239-edffa7f41984-kube-api-access-bhl8s\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:24:54.969196 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.969190 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:24:54.969196 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:54.969200 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c8abd1b-3c5d-410e-a239-edffa7f41984-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:24:55.587797 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:55.587769 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" Aug 05 17:24:55.587942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:55.587768 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9p9f8" event={"ID":"0c8abd1b-3c5d-410e-a239-edffa7f41984","Type":"ContainerDied","Data":"86fe6e1b67dfd6020bd23f47be158fcc9160a480fb70bb423a4a5aa89f842d2f"} Aug 05 17:24:55.587942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:24:55.587891 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86fe6e1b67dfd6020bd23f47be158fcc9160a480fb70bb423a4a5aa89f842d2f" Aug 05 17:25:01.145660 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.145629 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl"] Aug 05 17:25:01.146115 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146087 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="util" Aug 05 17:25:01.146115 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146106 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="util" Aug 05 17:25:01.146224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146129 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="pull" Aug 05 17:25:01.146224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146170 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="pull" Aug 05 17:25:01.146224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146207 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="extract" Aug 05 17:25:01.146224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146215 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="extract" Aug 05 17:25:01.146357 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.146330 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c8abd1b-3c5d-410e-a239-edffa7f41984" containerName="extract" Aug 05 17:25:01.148287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.148268 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.150517 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.150498 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 05 17:25:01.150620 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.150576 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 05 17:25:01.151446 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.151429 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rrtnf\"" Aug 05 17:25:01.156840 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.156819 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl"] Aug 05 17:25:01.220941 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.220910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z72gm\" (UniqueName: \"kubernetes.io/projected/b968d3ee-dc99-4362-a7fb-dd9becdd1694-kube-api-access-z72gm\") pod \"openshift-lws-operator-fd8ccff4c-5rhfl\" (UID: \"b968d3ee-dc99-4362-a7fb-dd9becdd1694\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.221091 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.220954 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b968d3ee-dc99-4362-a7fb-dd9becdd1694-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5rhfl\" (UID: \"b968d3ee-dc99-4362-a7fb-dd9becdd1694\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.322421 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.322389 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z72gm\" (UniqueName: \"kubernetes.io/projected/b968d3ee-dc99-4362-a7fb-dd9becdd1694-kube-api-access-z72gm\") pod \"openshift-lws-operator-fd8ccff4c-5rhfl\" (UID: \"b968d3ee-dc99-4362-a7fb-dd9becdd1694\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.322421 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.322426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b968d3ee-dc99-4362-a7fb-dd9becdd1694-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5rhfl\" (UID: \"b968d3ee-dc99-4362-a7fb-dd9becdd1694\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.322767 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.322750 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b968d3ee-dc99-4362-a7fb-dd9becdd1694-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5rhfl\" (UID: \"b968d3ee-dc99-4362-a7fb-dd9becdd1694\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.330458 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.330431 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z72gm\" (UniqueName: \"kubernetes.io/projected/b968d3ee-dc99-4362-a7fb-dd9becdd1694-kube-api-access-z72gm\") pod \"openshift-lws-operator-fd8ccff4c-5rhfl\" (UID: \"b968d3ee-dc99-4362-a7fb-dd9becdd1694\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.457220 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.457151 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" Aug 05 17:25:01.579203 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.579175 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl"] Aug 05 17:25:01.581135 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:25:01.581109 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb968d3ee_dc99_4362_a7fb_dd9becdd1694.slice/crio-038589b525055bf2fcd660cf08eec7334d0276f55127c48c636af1108f9f7b3c WatchSource:0}: Error finding container 038589b525055bf2fcd660cf08eec7334d0276f55127c48c636af1108f9f7b3c: Status 404 returned error can't find the container with id 038589b525055bf2fcd660cf08eec7334d0276f55127c48c636af1108f9f7b3c Aug 05 17:25:01.616747 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:01.616721 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" event={"ID":"b968d3ee-dc99-4362-a7fb-dd9becdd1694","Type":"ContainerStarted","Data":"038589b525055bf2fcd660cf08eec7334d0276f55127c48c636af1108f9f7b3c"} Aug 05 17:25:03.628715 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:03.628673 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" event={"ID":"b968d3ee-dc99-4362-a7fb-dd9becdd1694","Type":"ContainerStarted","Data":"6f3f528cd21f923d7ba37f015e2df232f5a56c5b938889a22e207b8ca9d71035"} Aug 05 17:25:03.645058 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:03.645002 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5rhfl" podStartSLOduration=0.703083309 podStartE2EDuration="2.644982703s" totalImagesPullingTime="1.941899394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:03.642920883 +0000 UTC m=+341.044149696" watchObservedRunningTime="2026-08-05 17:25:03.644982703 +0000 UTC m=+341.046211519" Aug 05 17:25:06.686153 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.686124 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2"] Aug 05 17:25:06.688676 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.688660 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.690695 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.690663 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 05 17:25:06.691450 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.691430 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 05 17:25:06.691593 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.691466 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lw45v\"" Aug 05 17:25:06.697579 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.697559 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2"] Aug 05 17:25:06.773730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.773692 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.773899 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.773779 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbczr\" (UniqueName: \"kubernetes.io/projected/60ca8066-1882-4b1b-8407-bfc5d27cf71c-kube-api-access-xbczr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.773899 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.773810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.874292 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.874238 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbczr\" (UniqueName: \"kubernetes.io/projected/60ca8066-1882-4b1b-8407-bfc5d27cf71c-kube-api-access-xbczr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.874477 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.874306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.874477 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.874370 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.874744 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.874720 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.874812 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.874752 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.881581 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.881560 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbczr\" (UniqueName: \"kubernetes.io/projected/60ca8066-1882-4b1b-8407-bfc5d27cf71c-kube-api-access-xbczr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:06.998789 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:06.998690 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:07.123037 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:07.123013 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2"] Aug 05 17:25:07.124912 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:25:07.124888 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ca8066_1882_4b1b_8407_bfc5d27cf71c.slice/crio-8b018352b3baa722bae6a2a9209b295a6e9eaea4f9dc35c13ff16edc0e51ff66 WatchSource:0}: Error finding container 8b018352b3baa722bae6a2a9209b295a6e9eaea4f9dc35c13ff16edc0e51ff66: Status 404 returned error can't find the container with id 8b018352b3baa722bae6a2a9209b295a6e9eaea4f9dc35c13ff16edc0e51ff66 Aug 05 17:25:07.648191 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:07.648155 2575 generic.go:356] "Generic (PLEG): container finished" podID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerID="9295c0d319b9567988174e8b7fb1cd976ee99aa2b5e8208f94bf9ecb82009f27" exitCode=0 Aug 05 17:25:07.648383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:07.648243 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" event={"ID":"60ca8066-1882-4b1b-8407-bfc5d27cf71c","Type":"ContainerDied","Data":"9295c0d319b9567988174e8b7fb1cd976ee99aa2b5e8208f94bf9ecb82009f27"} Aug 05 17:25:07.648383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:07.648302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" event={"ID":"60ca8066-1882-4b1b-8407-bfc5d27cf71c","Type":"ContainerStarted","Data":"8b018352b3baa722bae6a2a9209b295a6e9eaea4f9dc35c13ff16edc0e51ff66"} Aug 05 17:25:08.654605 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:08.654573 2575 generic.go:356] "Generic (PLEG): container finished" podID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerID="0ba3efed3a6b923ebf8b11d719a09d0ba7419ff4b76e689d232b934b64dc31f6" exitCode=0 Aug 05 17:25:08.654979 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:08.654657 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" event={"ID":"60ca8066-1882-4b1b-8407-bfc5d27cf71c","Type":"ContainerDied","Data":"0ba3efed3a6b923ebf8b11d719a09d0ba7419ff4b76e689d232b934b64dc31f6"} Aug 05 17:25:09.661332 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:09.661299 2575 generic.go:356] "Generic (PLEG): container finished" podID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerID="af9c7dfe105e395041680a9678e2a698eb888fa495372201021fd263a86e90df" exitCode=0 Aug 05 17:25:09.661802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:09.661390 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" event={"ID":"60ca8066-1882-4b1b-8407-bfc5d27cf71c","Type":"ContainerDied","Data":"af9c7dfe105e395041680a9678e2a698eb888fa495372201021fd263a86e90df"} Aug 05 17:25:10.787474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.787453 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:10.920731 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.920648 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-util\") pod \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " Aug 05 17:25:10.920731 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.920700 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-bundle\") pod \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " Aug 05 17:25:10.920947 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.920784 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbczr\" (UniqueName: \"kubernetes.io/projected/60ca8066-1882-4b1b-8407-bfc5d27cf71c-kube-api-access-xbczr\") pod \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\" (UID: \"60ca8066-1882-4b1b-8407-bfc5d27cf71c\") " Aug 05 17:25:10.921417 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.921385 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-bundle" (OuterVolumeSpecName: "bundle") pod "60ca8066-1882-4b1b-8407-bfc5d27cf71c" (UID: "60ca8066-1882-4b1b-8407-bfc5d27cf71c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:25:10.922859 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.922837 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ca8066-1882-4b1b-8407-bfc5d27cf71c-kube-api-access-xbczr" (OuterVolumeSpecName: "kube-api-access-xbczr") pod "60ca8066-1882-4b1b-8407-bfc5d27cf71c" (UID: "60ca8066-1882-4b1b-8407-bfc5d27cf71c"). InnerVolumeSpecName "kube-api-access-xbczr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:25:10.926009 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:10.925988 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-util" (OuterVolumeSpecName: "util") pod "60ca8066-1882-4b1b-8407-bfc5d27cf71c" (UID: "60ca8066-1882-4b1b-8407-bfc5d27cf71c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:25:11.021334 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:11.021310 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:25:11.021464 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:11.021413 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbczr\" (UniqueName: \"kubernetes.io/projected/60ca8066-1882-4b1b-8407-bfc5d27cf71c-kube-api-access-xbczr\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:25:11.021511 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:11.021488 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60ca8066-1882-4b1b-8407-bfc5d27cf71c-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:25:11.672487 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:11.672448 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" event={"ID":"60ca8066-1882-4b1b-8407-bfc5d27cf71c","Type":"ContainerDied","Data":"8b018352b3baa722bae6a2a9209b295a6e9eaea4f9dc35c13ff16edc0e51ff66"} Aug 05 17:25:11.672487 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:11.672465 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6rjf2" Aug 05 17:25:11.672487 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:11.672493 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b018352b3baa722bae6a2a9209b295a6e9eaea4f9dc35c13ff16edc0e51ff66" Aug 05 17:25:23.163992 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.163922 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr"] Aug 05 17:25:23.164364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164272 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="pull" Aug 05 17:25:23.164364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164285 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="pull" Aug 05 17:25:23.164364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164294 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="util" Aug 05 17:25:23.164364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164299 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="util" Aug 05 17:25:23.164364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164306 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="extract" Aug 05 17:25:23.164364 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164311 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="extract" Aug 05 17:25:23.164802 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.164376 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ca8066-1882-4b1b-8407-bfc5d27cf71c" containerName="extract" Aug 05 17:25:23.170488 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.170466 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.172494 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.172472 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9vhvx\"" Aug 05 17:25:23.172494 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.172478 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 05 17:25:23.172783 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.172769 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 05 17:25:23.172783 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.172779 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 05 17:25:23.172995 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.172980 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 05 17:25:23.185115 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.185096 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr"] Aug 05 17:25:23.330290 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.330253 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a5356c-a479-420a-9906-5c9f54e22d91-webhook-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.330463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.330341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a5356c-a479-420a-9906-5c9f54e22d91-tmp\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.330463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.330379 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6jx6\" (UniqueName: \"kubernetes.io/projected/98a5356c-a479-420a-9906-5c9f54e22d91-kube-api-access-t6jx6\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.330463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.330419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/98a5356c-a479-420a-9906-5c9f54e22d91-manifests\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.330568 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.330499 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/98a5356c-a479-420a-9906-5c9f54e22d91-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.431514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.431435 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a5356c-a479-420a-9906-5c9f54e22d91-tmp\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.431514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.431471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6jx6\" (UniqueName: \"kubernetes.io/projected/98a5356c-a479-420a-9906-5c9f54e22d91-kube-api-access-t6jx6\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.431514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.431494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/98a5356c-a479-420a-9906-5c9f54e22d91-manifests\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.431771 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.431521 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/98a5356c-a479-420a-9906-5c9f54e22d91-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.431771 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.431594 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a5356c-a479-420a-9906-5c9f54e22d91-webhook-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.431948 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.431924 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/98a5356c-a479-420a-9906-5c9f54e22d91-manifests\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.433636 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.433611 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a5356c-a479-420a-9906-5c9f54e22d91-tmp\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.433855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.433838 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/98a5356c-a479-420a-9906-5c9f54e22d91-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.433897 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.433868 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98a5356c-a479-420a-9906-5c9f54e22d91-webhook-cert\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.438984 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.438952 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6jx6\" (UniqueName: \"kubernetes.io/projected/98a5356c-a479-420a-9906-5c9f54e22d91-kube-api-access-t6jx6\") pod \"opendatahub-operator-controller-manager-58c579b484-27rsr\" (UID: \"98a5356c-a479-420a-9906-5c9f54e22d91\") " pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.480779 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.480755 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:23.612457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.612432 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr"] Aug 05 17:25:23.614601 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:25:23.614575 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a5356c_a479_420a_9906_5c9f54e22d91.slice/crio-b65a5c9dd806c0548d80fd42a226b3590de6cad3530599e6bc8960ec58800416 WatchSource:0}: Error finding container b65a5c9dd806c0548d80fd42a226b3590de6cad3530599e6bc8960ec58800416: Status 404 returned error can't find the container with id b65a5c9dd806c0548d80fd42a226b3590de6cad3530599e6bc8960ec58800416 Aug 05 17:25:23.734630 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:23.734537 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" event={"ID":"98a5356c-a479-420a-9906-5c9f54e22d91","Type":"ContainerStarted","Data":"b65a5c9dd806c0548d80fd42a226b3590de6cad3530599e6bc8960ec58800416"} Aug 05 17:25:27.759625 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:27.759585 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" event={"ID":"98a5356c-a479-420a-9906-5c9f54e22d91","Type":"ContainerStarted","Data":"93d0a93617029dee23b894e70bb483770b619b6e356efed13783fece176eb345"} Aug 05 17:25:28.766289 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:28.766259 2575 generic.go:356] "Generic (PLEG): container finished" podID="98a5356c-a479-420a-9906-5c9f54e22d91" containerID="93d0a93617029dee23b894e70bb483770b619b6e356efed13783fece176eb345" exitCode=0 Aug 05 17:25:28.766700 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:28.766336 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" event={"ID":"98a5356c-a479-420a-9906-5c9f54e22d91","Type":"ContainerDied","Data":"93d0a93617029dee23b894e70bb483770b619b6e356efed13783fece176eb345"} Aug 05 17:25:29.773008 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:29.772964 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" event={"ID":"98a5356c-a479-420a-9906-5c9f54e22d91","Type":"ContainerStarted","Data":"cbf2b4acffbc504766d4496288233acab8bd20f5ebbc25968f0e9fc481ef4d9c"} Aug 05 17:25:29.773464 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:29.773109 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:29.795590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:29.795535 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" podStartSLOduration=2.212242273 podStartE2EDuration="6.795520125s" totalImagesPullingTime="3.557567498s" totalInitContainerRuntime="1.025710354s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:29.79432865 +0000 UTC m=+367.195557464" watchObservedRunningTime="2026-08-05 17:25:29.795520125 +0000 UTC m=+367.196748938" Aug 05 17:25:40.780535 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:40.780505 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58c579b484-27rsr" Aug 05 17:25:43.256713 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.256677 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg"] Aug 05 17:25:43.265300 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.265275 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.268727 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.268699 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 05 17:25:43.268855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.268771 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 05 17:25:43.269554 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.269532 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Aug 05 17:25:43.269727 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.269650 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8ttfc\"" Aug 05 17:25:43.269842 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.269735 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Aug 05 17:25:43.278754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.278730 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg"] Aug 05 17:25:43.304643 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.304616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/496be72c-9be6-4d1a-a7ce-0f05028363a9-tmp\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.304796 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.304672 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/496be72c-9be6-4d1a-a7ce-0f05028363a9-tls-certs\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.304796 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.304718 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtzc5\" (UniqueName: \"kubernetes.io/projected/496be72c-9be6-4d1a-a7ce-0f05028363a9-kube-api-access-wtzc5\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.406090 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.406030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/496be72c-9be6-4d1a-a7ce-0f05028363a9-tls-certs\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.406270 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.406118 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtzc5\" (UniqueName: \"kubernetes.io/projected/496be72c-9be6-4d1a-a7ce-0f05028363a9-kube-api-access-wtzc5\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.406270 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.406209 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/496be72c-9be6-4d1a-a7ce-0f05028363a9-tmp\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.408480 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.408448 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/496be72c-9be6-4d1a-a7ce-0f05028363a9-tmp\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.408760 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.408736 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/496be72c-9be6-4d1a-a7ce-0f05028363a9-tls-certs\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.426397 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.426372 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtzc5\" (UniqueName: \"kubernetes.io/projected/496be72c-9be6-4d1a-a7ce-0f05028363a9-kube-api-access-wtzc5\") pod \"kube-auth-proxy-7c99fc649-hvhzg\" (UID: \"496be72c-9be6-4d1a-a7ce-0f05028363a9\") " pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.575520 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.575491 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" Aug 05 17:25:43.753620 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.753583 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg"] Aug 05 17:25:43.756471 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:25:43.756441 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496be72c_9be6_4d1a_a7ce_0f05028363a9.slice/crio-6678ae40eba8d1d8a814419466b0d9acf26e85cffc7e0ce9234f29011a5d7079 WatchSource:0}: Error finding container 6678ae40eba8d1d8a814419466b0d9acf26e85cffc7e0ce9234f29011a5d7079: Status 404 returned error can't find the container with id 6678ae40eba8d1d8a814419466b0d9acf26e85cffc7e0ce9234f29011a5d7079 Aug 05 17:25:43.831435 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.831365 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw"] Aug 05 17:25:43.836138 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.836119 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:43.841045 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.841015 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" event={"ID":"496be72c-9be6-4d1a-a7ce-0f05028363a9","Type":"ContainerStarted","Data":"6678ae40eba8d1d8a814419466b0d9acf26e85cffc7e0ce9234f29011a5d7079"} Aug 05 17:25:43.844833 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.844816 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 05 17:25:43.845375 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.845360 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4hnvq\"" Aug 05 17:25:43.845521 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.845504 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 05 17:25:43.845597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.845525 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 05 17:25:43.886948 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.886919 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw"] Aug 05 17:25:43.910406 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.910376 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dffd07d5-019b-44db-9707-01ca6c262890-cert\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:43.910526 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.910411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dffd07d5-019b-44db-9707-01ca6c262890-metrics-cert\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:43.910526 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.910434 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dffd07d5-019b-44db-9707-01ca6c262890-manager-config\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:43.910626 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:43.910550 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn5b8\" (UniqueName: \"kubernetes.io/projected/dffd07d5-019b-44db-9707-01ca6c262890-kube-api-access-zn5b8\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.011871 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.011838 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn5b8\" (UniqueName: \"kubernetes.io/projected/dffd07d5-019b-44db-9707-01ca6c262890-kube-api-access-zn5b8\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.012087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.012001 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dffd07d5-019b-44db-9707-01ca6c262890-cert\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.012087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.012035 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dffd07d5-019b-44db-9707-01ca6c262890-metrics-cert\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.012087 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.012081 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dffd07d5-019b-44db-9707-01ca6c262890-manager-config\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.012776 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.012757 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dffd07d5-019b-44db-9707-01ca6c262890-manager-config\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.014399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.014381 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dffd07d5-019b-44db-9707-01ca6c262890-cert\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.014450 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.014403 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dffd07d5-019b-44db-9707-01ca6c262890-metrics-cert\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.031454 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.031431 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn5b8\" (UniqueName: \"kubernetes.io/projected/dffd07d5-019b-44db-9707-01ca6c262890-kube-api-access-zn5b8\") pod \"lws-controller-manager-57fb49d769-xk5sw\" (UID: \"dffd07d5-019b-44db-9707-01ca6c262890\") " pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.146168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.146032 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:44.290572 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.290547 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw"] Aug 05 17:25:44.291857 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:25:44.291832 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddffd07d5_019b_44db_9707_01ca6c262890.slice/crio-5672872e4041fb51fa28812fc3ba989b230c9c883319b6915cce571130863865 WatchSource:0}: Error finding container 5672872e4041fb51fa28812fc3ba989b230c9c883319b6915cce571130863865: Status 404 returned error can't find the container with id 5672872e4041fb51fa28812fc3ba989b230c9c883319b6915cce571130863865 Aug 05 17:25:44.848250 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:44.848207 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" event={"ID":"dffd07d5-019b-44db-9707-01ca6c262890","Type":"ContainerStarted","Data":"5672872e4041fb51fa28812fc3ba989b230c9c883319b6915cce571130863865"} Aug 05 17:25:47.864756 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:47.864715 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" event={"ID":"dffd07d5-019b-44db-9707-01ca6c262890","Type":"ContainerStarted","Data":"11385bb6dfa6970692fea39dd144eb5793438330699cd6f8d0b539d40e08ddf8"} Aug 05 17:25:47.865245 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:47.864810 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:25:47.866219 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:47.866199 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" event={"ID":"496be72c-9be6-4d1a-a7ce-0f05028363a9","Type":"ContainerStarted","Data":"7775a6ce153e180fbbe56e2e5da4667f16cd02d837ed92cd9d385de9e3488dcc"} Aug 05 17:25:47.883902 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:47.883854 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" podStartSLOduration=2.028898759 podStartE2EDuration="4.883836688s" totalImagesPullingTime="2.854937929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:47.882440135 +0000 UTC m=+385.283668944" watchObservedRunningTime="2026-08-05 17:25:47.883836688 +0000 UTC m=+385.285065505" Aug 05 17:25:47.905469 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:47.905426 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c99fc649-hvhzg" podStartSLOduration=1.511587937 podStartE2EDuration="4.905413926s" totalImagesPullingTime="3.393825989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:25:47.903635781 +0000 UTC m=+385.304864595" watchObservedRunningTime="2026-08-05 17:25:47.905413926 +0000 UTC m=+385.306642739" Aug 05 17:25:58.873374 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:25:58.873344 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57fb49d769-xk5sw" Aug 05 17:26:28.421461 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.421385 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f5668dc8d-d28w2"] Aug 05 17:26:28.427392 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.427344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.436508 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.435848 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5668dc8d-d28w2"] Aug 05 17:26:28.506409 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c99278-a570-4aa6-8820-1ad239713bb2-console-oauth-config\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.506590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlkcm\" (UniqueName: \"kubernetes.io/projected/18c99278-a570-4aa6-8820-1ad239713bb2-kube-api-access-tlkcm\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.506590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506444 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-oauth-serving-cert\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.506590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506519 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c99278-a570-4aa6-8820-1ad239713bb2-console-serving-cert\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.506590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506569 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-console-config\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.506590 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-trusted-ca-bundle\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.506816 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.506675 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-service-ca\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607110 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-oauth-serving-cert\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607290 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607129 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c99278-a570-4aa6-8820-1ad239713bb2-console-serving-cert\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607290 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607159 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-console-config\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607290 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-trusted-ca-bundle\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607290 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-service-ca\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607290 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607283 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c99278-a570-4aa6-8820-1ad239713bb2-console-oauth-config\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607555 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607314 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlkcm\" (UniqueName: \"kubernetes.io/projected/18c99278-a570-4aa6-8820-1ad239713bb2-kube-api-access-tlkcm\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607828 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607802 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-oauth-serving-cert\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607934 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607833 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-console-config\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.607985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.607963 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-service-ca\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.608162 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.608138 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18c99278-a570-4aa6-8820-1ad239713bb2-trusted-ca-bundle\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.609687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.609666 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/18c99278-a570-4aa6-8820-1ad239713bb2-console-oauth-config\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.609772 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.609754 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/18c99278-a570-4aa6-8820-1ad239713bb2-console-serving-cert\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.615155 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.615134 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlkcm\" (UniqueName: \"kubernetes.io/projected/18c99278-a570-4aa6-8820-1ad239713bb2-kube-api-access-tlkcm\") pod \"console-6f5668dc8d-d28w2\" (UID: \"18c99278-a570-4aa6-8820-1ad239713bb2\") " pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.739510 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.739425 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:28.877844 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:28.877770 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5668dc8d-d28w2"] Aug 05 17:26:28.880065 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:26:28.880023 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c99278_a570_4aa6_8820_1ad239713bb2.slice/crio-bb67c67a9ba4b67e9ff38ba20711b970fac33627e0f03dee1d7bfaff99efa5a4 WatchSource:0}: Error finding container bb67c67a9ba4b67e9ff38ba20711b970fac33627e0f03dee1d7bfaff99efa5a4: Status 404 returned error can't find the container with id bb67c67a9ba4b67e9ff38ba20711b970fac33627e0f03dee1d7bfaff99efa5a4 Aug 05 17:26:29.070659 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:29.070619 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5668dc8d-d28w2" event={"ID":"18c99278-a570-4aa6-8820-1ad239713bb2","Type":"ContainerStarted","Data":"5c03eab84e41bb22d32930c0c3644950bd82d84f596fd418de47d415ab85683a"} Aug 05 17:26:29.070808 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:29.070672 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5668dc8d-d28w2" event={"ID":"18c99278-a570-4aa6-8820-1ad239713bb2","Type":"ContainerStarted","Data":"bb67c67a9ba4b67e9ff38ba20711b970fac33627e0f03dee1d7bfaff99efa5a4"} Aug 05 17:26:29.090631 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:29.090582 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f5668dc8d-d28w2" podStartSLOduration=1.090567095 podStartE2EDuration="1.090567095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:26:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:26:29.08929016 +0000 UTC m=+426.490518972" watchObservedRunningTime="2026-08-05 17:26:29.090567095 +0000 UTC m=+426.491795909" Aug 05 17:26:38.740208 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:38.740169 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:38.740772 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:38.740232 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:38.744826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:38.744804 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:39.119188 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:39.119161 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f5668dc8d-d28w2" Aug 05 17:26:39.161386 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:39.161352 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bddf6b9f7-nbxrr"] Aug 05 17:26:47.815655 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.815574 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw"] Aug 05 17:26:47.818890 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.818871 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.821132 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.821104 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 05 17:26:47.821246 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.821152 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lw45v\"" Aug 05 17:26:47.821724 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.821709 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 05 17:26:47.826676 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.826533 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw"] Aug 05 17:26:47.875573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.875543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.875730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.875582 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkhjc\" (UniqueName: \"kubernetes.io/projected/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-kube-api-access-lkhjc\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.875730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.875670 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.910518 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.910488 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq"] Aug 05 17:26:47.913621 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.913605 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:47.922941 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.922911 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq"] Aug 05 17:26:47.976162 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976127 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.976162 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976160 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhjc\" (UniqueName: \"kubernetes.io/projected/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-kube-api-access-lkhjc\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.976365 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976204 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lbdc\" (UniqueName: \"kubernetes.io/projected/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-kube-api-access-5lbdc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:47.976907 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.976907 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976783 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:47.976907 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:47.977166 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.976981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.978006 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.977496 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:47.986677 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:47.986653 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkhjc\" (UniqueName: \"kubernetes.io/projected/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-kube-api-access-lkhjc\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:48.014129 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.014099 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd"] Aug 05 17:26:48.017542 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.017526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.025892 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.025864 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd"] Aug 05 17:26:48.077849 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.077780 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.077849 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.077830 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lbdc\" (UniqueName: \"kubernetes.io/projected/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-kube-api-access-5lbdc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.077849 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.077849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.078031 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.077901 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.078031 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.077943 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.078031 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.078003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72c9h\" (UniqueName: \"kubernetes.io/projected/9ef5d311-3675-459b-a82d-8690d618bab2-kube-api-access-72c9h\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.078267 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.078250 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.078365 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.078349 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.085357 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.085334 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lbdc\" (UniqueName: \"kubernetes.io/projected/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-kube-api-access-5lbdc\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.113291 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.113270 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n"] Aug 05 17:26:48.116688 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.116674 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.124800 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.124782 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n"] Aug 05 17:26:48.129490 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.129471 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:48.179331 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.179256 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcjq4\" (UniqueName: \"kubernetes.io/projected/499ffb1c-806c-4dd1-b37b-a5b97b74695f-kube-api-access-jcjq4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.179429 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.179336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.179486 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.179453 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72c9h\" (UniqueName: \"kubernetes.io/projected/9ef5d311-3675-459b-a82d-8690d618bab2-kube-api-access-72c9h\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.179549 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.179502 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.179601 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.179577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.179650 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.179607 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.180274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.180245 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.180274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.180265 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.188420 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.188390 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72c9h\" (UniqueName: \"kubernetes.io/projected/9ef5d311-3675-459b-a82d-8690d618bab2-kube-api-access-72c9h\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.223634 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.223600 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:48.262487 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.262463 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw"] Aug 05 17:26:48.264351 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:26:48.264320 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141b72a3_e18e_49e9_8ca1_f97497aaa1b3.slice/crio-56ef7fd5e10e8b4aa40b1d07ebab997d7c4e138bb14dc264de7f096751056253 WatchSource:0}: Error finding container 56ef7fd5e10e8b4aa40b1d07ebab997d7c4e138bb14dc264de7f096751056253: Status 404 returned error can't find the container with id 56ef7fd5e10e8b4aa40b1d07ebab997d7c4e138bb14dc264de7f096751056253 Aug 05 17:26:48.280759 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.280702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.281180 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.281153 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.281291 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.281176 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.281359 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.281331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcjq4\" (UniqueName: \"kubernetes.io/projected/499ffb1c-806c-4dd1-b37b-a5b97b74695f-kube-api-access-jcjq4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.281521 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.281492 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.291494 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.291473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcjq4\" (UniqueName: \"kubernetes.io/projected/499ffb1c-806c-4dd1-b37b-a5b97b74695f-kube-api-access-jcjq4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.327812 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.327631 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:48.369186 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.369147 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq"] Aug 05 17:26:48.372476 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:26:48.372444 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ce79f0_08f9_4c3a_a51d_e7a36a79bb98.slice/crio-bfbb72bdd7e0e4493c958847bffad4513bbef8f4c4c529057a4eb54696aa694b WatchSource:0}: Error finding container bfbb72bdd7e0e4493c958847bffad4513bbef8f4c4c529057a4eb54696aa694b: Status 404 returned error can't find the container with id bfbb72bdd7e0e4493c958847bffad4513bbef8f4c4c529057a4eb54696aa694b Aug 05 17:26:48.426795 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.426766 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:48.470355 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.470325 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd"] Aug 05 17:26:48.471677 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:26:48.471627 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef5d311_3675_459b_a82d_8690d618bab2.slice/crio-2089d3b42bc4fcf15b6832b511b9671063b79073b8dbd586d961c71b16701130 WatchSource:0}: Error finding container 2089d3b42bc4fcf15b6832b511b9671063b79073b8dbd586d961c71b16701130: Status 404 returned error can't find the container with id 2089d3b42bc4fcf15b6832b511b9671063b79073b8dbd586d961c71b16701130 Aug 05 17:26:48.564129 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:48.564104 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n"] Aug 05 17:26:48.569276 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:26:48.566823 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499ffb1c_806c_4dd1_b37b_a5b97b74695f.slice/crio-e05cf14b499919ab970e128aaee281e9601c91cc1b3a3261d9bde29230702f13 WatchSource:0}: Error finding container e05cf14b499919ab970e128aaee281e9601c91cc1b3a3261d9bde29230702f13: Status 404 returned error can't find the container with id e05cf14b499919ab970e128aaee281e9601c91cc1b3a3261d9bde29230702f13 Aug 05 17:26:49.168538 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.168501 2575 generic.go:356] "Generic (PLEG): container finished" podID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerID="dbecd4f40d7714b2da398ba879123090649877775c9d4064d1f2c8cfc9675cbd" exitCode=0 Aug 05 17:26:49.168998 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.168584 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" event={"ID":"499ffb1c-806c-4dd1-b37b-a5b97b74695f","Type":"ContainerDied","Data":"dbecd4f40d7714b2da398ba879123090649877775c9d4064d1f2c8cfc9675cbd"} Aug 05 17:26:49.168998 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.168629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" event={"ID":"499ffb1c-806c-4dd1-b37b-a5b97b74695f","Type":"ContainerStarted","Data":"e05cf14b499919ab970e128aaee281e9601c91cc1b3a3261d9bde29230702f13"} Aug 05 17:26:49.170241 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.170219 2575 generic.go:356] "Generic (PLEG): container finished" podID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerID="bb6223184526d9f610e898daef4eace6202dd3a4b3a3b1d66ca13b2db85041be" exitCode=0 Aug 05 17:26:49.170317 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.170261 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" event={"ID":"141b72a3-e18e-49e9-8ca1-f97497aaa1b3","Type":"ContainerDied","Data":"bb6223184526d9f610e898daef4eace6202dd3a4b3a3b1d66ca13b2db85041be"} Aug 05 17:26:49.170317 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.170310 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" event={"ID":"141b72a3-e18e-49e9-8ca1-f97497aaa1b3","Type":"ContainerStarted","Data":"56ef7fd5e10e8b4aa40b1d07ebab997d7c4e138bb14dc264de7f096751056253"} Aug 05 17:26:49.171800 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.171782 2575 generic.go:356] "Generic (PLEG): container finished" podID="9ef5d311-3675-459b-a82d-8690d618bab2" containerID="081f101003d9bc0937426d182558fc0620a6371f998a333ccbef8b134d34f3ca" exitCode=0 Aug 05 17:26:49.171875 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.171854 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" event={"ID":"9ef5d311-3675-459b-a82d-8690d618bab2","Type":"ContainerDied","Data":"081f101003d9bc0937426d182558fc0620a6371f998a333ccbef8b134d34f3ca"} Aug 05 17:26:49.172194 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.171899 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" event={"ID":"9ef5d311-3675-459b-a82d-8690d618bab2","Type":"ContainerStarted","Data":"2089d3b42bc4fcf15b6832b511b9671063b79073b8dbd586d961c71b16701130"} Aug 05 17:26:49.173390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.173373 2575 generic.go:356] "Generic (PLEG): container finished" podID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerID="b0d499c9fb8c4f3ef057f99e44f408da43049c23ff3c6560eb98b9d57cacd283" exitCode=0 Aug 05 17:26:49.173483 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.173413 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" event={"ID":"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98","Type":"ContainerDied","Data":"b0d499c9fb8c4f3ef057f99e44f408da43049c23ff3c6560eb98b9d57cacd283"} Aug 05 17:26:49.173483 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:49.173438 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" event={"ID":"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98","Type":"ContainerStarted","Data":"bfbb72bdd7e0e4493c958847bffad4513bbef8f4c4c529057a4eb54696aa694b"} Aug 05 17:26:50.181356 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:50.181331 2575 generic.go:356] "Generic (PLEG): container finished" podID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerID="f51c1117a7d94a6f5e0e1748baad1fe4c221c44d541c204dc804fedac00d261b" exitCode=0 Aug 05 17:26:50.181683 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:50.181424 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" event={"ID":"141b72a3-e18e-49e9-8ca1-f97497aaa1b3","Type":"ContainerDied","Data":"f51c1117a7d94a6f5e0e1748baad1fe4c221c44d541c204dc804fedac00d261b"} Aug 05 17:26:51.188111 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.188071 2575 generic.go:356] "Generic (PLEG): container finished" podID="9ef5d311-3675-459b-a82d-8690d618bab2" containerID="4909b58e6af679009195bf7054a47df4cfc0912fa6d5841ed0a415c6b8ed27f6" exitCode=0 Aug 05 17:26:51.188567 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.188144 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" event={"ID":"9ef5d311-3675-459b-a82d-8690d618bab2","Type":"ContainerDied","Data":"4909b58e6af679009195bf7054a47df4cfc0912fa6d5841ed0a415c6b8ed27f6"} Aug 05 17:26:51.190022 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.189998 2575 generic.go:356] "Generic (PLEG): container finished" podID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerID="3d5e906c596c79b8ebe83b0008c0ec2030cdc67ae6255256550491dcaaca0802" exitCode=0 Aug 05 17:26:51.190101 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.190003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" event={"ID":"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98","Type":"ContainerDied","Data":"3d5e906c596c79b8ebe83b0008c0ec2030cdc67ae6255256550491dcaaca0802"} Aug 05 17:26:51.191994 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.191977 2575 generic.go:356] "Generic (PLEG): container finished" podID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerID="3c7daadb62952269c53f6f384307570b49642d26a0c136cf629aea9092f5f407" exitCode=0 Aug 05 17:26:51.192092 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.192063 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" event={"ID":"499ffb1c-806c-4dd1-b37b-a5b97b74695f","Type":"ContainerDied","Data":"3c7daadb62952269c53f6f384307570b49642d26a0c136cf629aea9092f5f407"} Aug 05 17:26:51.194145 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.194119 2575 generic.go:356] "Generic (PLEG): container finished" podID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerID="78c58dc35a878f32c2d057b0190fc43408367a78e3720916eeb5da7165016384" exitCode=0 Aug 05 17:26:51.194211 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:51.194155 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" event={"ID":"141b72a3-e18e-49e9-8ca1-f97497aaa1b3","Type":"ContainerDied","Data":"78c58dc35a878f32c2d057b0190fc43408367a78e3720916eeb5da7165016384"} Aug 05 17:26:52.203341 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.203308 2575 generic.go:356] "Generic (PLEG): container finished" podID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerID="31ee0fdef9ad64a27ca82365bfa5196677819ca105e3732209d1649030532dba" exitCode=0 Aug 05 17:26:52.203786 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.203391 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" event={"ID":"499ffb1c-806c-4dd1-b37b-a5b97b74695f","Type":"ContainerDied","Data":"31ee0fdef9ad64a27ca82365bfa5196677819ca105e3732209d1649030532dba"} Aug 05 17:26:52.205344 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.205326 2575 generic.go:356] "Generic (PLEG): container finished" podID="9ef5d311-3675-459b-a82d-8690d618bab2" containerID="f1fc9196baa42020d2d6ffe8ce665645eb6a05f01e8089df589841c0c26a7a6b" exitCode=0 Aug 05 17:26:52.205438 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.205386 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" event={"ID":"9ef5d311-3675-459b-a82d-8690d618bab2","Type":"ContainerDied","Data":"f1fc9196baa42020d2d6ffe8ce665645eb6a05f01e8089df589841c0c26a7a6b"} Aug 05 17:26:52.207113 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.207090 2575 generic.go:356] "Generic (PLEG): container finished" podID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerID="7be0459090686840791d2d605ca4d75e0367e3e03603b3df45a322be9e9b8a84" exitCode=0 Aug 05 17:26:52.207200 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.207117 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" event={"ID":"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98","Type":"ContainerDied","Data":"7be0459090686840791d2d605ca4d75e0367e3e03603b3df45a322be9e9b8a84"} Aug 05 17:26:52.340173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.340115 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:52.420604 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.420575 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-util\") pod \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " Aug 05 17:26:52.420779 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.420663 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkhjc\" (UniqueName: \"kubernetes.io/projected/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-kube-api-access-lkhjc\") pod \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " Aug 05 17:26:52.420779 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.420743 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-bundle\") pod \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\" (UID: \"141b72a3-e18e-49e9-8ca1-f97497aaa1b3\") " Aug 05 17:26:52.421287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.421253 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-bundle" (OuterVolumeSpecName: "bundle") pod "141b72a3-e18e-49e9-8ca1-f97497aaa1b3" (UID: "141b72a3-e18e-49e9-8ca1-f97497aaa1b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:52.422705 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.422683 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-kube-api-access-lkhjc" (OuterVolumeSpecName: "kube-api-access-lkhjc") pod "141b72a3-e18e-49e9-8ca1-f97497aaa1b3" (UID: "141b72a3-e18e-49e9-8ca1-f97497aaa1b3"). InnerVolumeSpecName "kube-api-access-lkhjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:26:52.426588 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.426565 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-util" (OuterVolumeSpecName: "util") pod "141b72a3-e18e-49e9-8ca1-f97497aaa1b3" (UID: "141b72a3-e18e-49e9-8ca1-f97497aaa1b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:52.521754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.521717 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:52.521754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.521746 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkhjc\" (UniqueName: \"kubernetes.io/projected/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-kube-api-access-lkhjc\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:52.521754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:52.521757 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/141b72a3-e18e-49e9-8ca1-f97497aaa1b3-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.213642 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.213612 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" Aug 05 17:26:53.213981 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.213651 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbpvdw" event={"ID":"141b72a3-e18e-49e9-8ca1-f97497aaa1b3","Type":"ContainerDied","Data":"56ef7fd5e10e8b4aa40b1d07ebab997d7c4e138bb14dc264de7f096751056253"} Aug 05 17:26:53.213981 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.213700 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56ef7fd5e10e8b4aa40b1d07ebab997d7c4e138bb14dc264de7f096751056253" Aug 05 17:26:53.365898 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.365872 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:26:53.402080 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.402042 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:53.405977 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.405957 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:53.429971 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.429947 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-bundle\") pod \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " Aug 05 17:26:53.430078 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.430010 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcjq4\" (UniqueName: \"kubernetes.io/projected/499ffb1c-806c-4dd1-b37b-a5b97b74695f-kube-api-access-jcjq4\") pod \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " Aug 05 17:26:53.430078 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.430042 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-util\") pod \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\" (UID: \"499ffb1c-806c-4dd1-b37b-a5b97b74695f\") " Aug 05 17:26:53.430495 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.430471 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-bundle" (OuterVolumeSpecName: "bundle") pod "499ffb1c-806c-4dd1-b37b-a5b97b74695f" (UID: "499ffb1c-806c-4dd1-b37b-a5b97b74695f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:53.432077 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.432032 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499ffb1c-806c-4dd1-b37b-a5b97b74695f-kube-api-access-jcjq4" (OuterVolumeSpecName: "kube-api-access-jcjq4") pod "499ffb1c-806c-4dd1-b37b-a5b97b74695f" (UID: "499ffb1c-806c-4dd1-b37b-a5b97b74695f"). InnerVolumeSpecName "kube-api-access-jcjq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:26:53.435472 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.435452 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-util" (OuterVolumeSpecName: "util") pod "499ffb1c-806c-4dd1-b37b-a5b97b74695f" (UID: "499ffb1c-806c-4dd1-b37b-a5b97b74695f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:53.531159 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531122 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-bundle\") pod \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " Aug 05 17:26:53.531327 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531180 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-util\") pod \"9ef5d311-3675-459b-a82d-8690d618bab2\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " Aug 05 17:26:53.531327 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531204 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-util\") pod \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " Aug 05 17:26:53.531327 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531236 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-bundle\") pod \"9ef5d311-3675-459b-a82d-8690d618bab2\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " Aug 05 17:26:53.531327 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531282 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lbdc\" (UniqueName: \"kubernetes.io/projected/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-kube-api-access-5lbdc\") pod \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\" (UID: \"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98\") " Aug 05 17:26:53.531327 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531316 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72c9h\" (UniqueName: \"kubernetes.io/projected/9ef5d311-3675-459b-a82d-8690d618bab2-kube-api-access-72c9h\") pod \"9ef5d311-3675-459b-a82d-8690d618bab2\" (UID: \"9ef5d311-3675-459b-a82d-8690d618bab2\") " Aug 05 17:26:53.531687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531622 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.531687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531641 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcjq4\" (UniqueName: \"kubernetes.io/projected/499ffb1c-806c-4dd1-b37b-a5b97b74695f-kube-api-access-jcjq4\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.531687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531653 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/499ffb1c-806c-4dd1-b37b-a5b97b74695f-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.531867 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531678 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-bundle" (OuterVolumeSpecName: "bundle") pod "10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" (UID: "10ce79f0-08f9-4c3a-a51d-e7a36a79bb98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:53.531968 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.531940 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-bundle" (OuterVolumeSpecName: "bundle") pod "9ef5d311-3675-459b-a82d-8690d618bab2" (UID: "9ef5d311-3675-459b-a82d-8690d618bab2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:53.533817 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.533790 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef5d311-3675-459b-a82d-8690d618bab2-kube-api-access-72c9h" (OuterVolumeSpecName: "kube-api-access-72c9h") pod "9ef5d311-3675-459b-a82d-8690d618bab2" (UID: "9ef5d311-3675-459b-a82d-8690d618bab2"). InnerVolumeSpecName "kube-api-access-72c9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:26:53.533884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.533828 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-kube-api-access-5lbdc" (OuterVolumeSpecName: "kube-api-access-5lbdc") pod "10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" (UID: "10ce79f0-08f9-4c3a-a51d-e7a36a79bb98"). InnerVolumeSpecName "kube-api-access-5lbdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:26:53.537120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.537074 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-util" (OuterVolumeSpecName: "util") pod "10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" (UID: "10ce79f0-08f9-4c3a-a51d-e7a36a79bb98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:53.537535 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.537515 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-util" (OuterVolumeSpecName: "util") pod "9ef5d311-3675-459b-a82d-8690d618bab2" (UID: "9ef5d311-3675-459b-a82d-8690d618bab2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:26:53.632884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.632858 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.632884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.632880 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-util\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.632884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.632889 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ef5d311-3675-459b-a82d-8690d618bab2-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.633096 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.632897 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lbdc\" (UniqueName: \"kubernetes.io/projected/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-kube-api-access-5lbdc\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.633096 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.632907 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72c9h\" (UniqueName: \"kubernetes.io/projected/9ef5d311-3675-459b-a82d-8690d618bab2-kube-api-access-72c9h\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:53.633096 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:53.632916 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10ce79f0-08f9-4c3a-a51d-e7a36a79bb98-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:26:54.220942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.220905 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" event={"ID":"9ef5d311-3675-459b-a82d-8690d618bab2","Type":"ContainerDied","Data":"2089d3b42bc4fcf15b6832b511b9671063b79073b8dbd586d961c71b16701130"} Aug 05 17:26:54.221390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.220960 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2089d3b42bc4fcf15b6832b511b9671063b79073b8dbd586d961c71b16701130" Aug 05 17:26:54.221390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.220926 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d870536p6jd" Aug 05 17:26:54.223016 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.222987 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" Aug 05 17:26:54.223138 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.223031 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb6h5lq" event={"ID":"10ce79f0-08f9-4c3a-a51d-e7a36a79bb98","Type":"ContainerDied","Data":"bfbb72bdd7e0e4493c958847bffad4513bbef8f4c4c529057a4eb54696aa694b"} Aug 05 17:26:54.223138 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.223097 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfbb72bdd7e0e4493c958847bffad4513bbef8f4c4c529057a4eb54696aa694b" Aug 05 17:26:54.225295 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.225265 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" event={"ID":"499ffb1c-806c-4dd1-b37b-a5b97b74695f","Type":"ContainerDied","Data":"e05cf14b499919ab970e128aaee281e9601c91cc1b3a3261d9bde29230702f13"} Aug 05 17:26:54.225417 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.225301 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e05cf14b499919ab970e128aaee281e9601c91cc1b3a3261d9bde29230702f13" Aug 05 17:26:54.225417 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:26:54.225359 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t8s9n" Aug 05 17:27:04.183133 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.183045 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bddf6b9f7-nbxrr" podUID="c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" containerName="console" containerID="cri-o://d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835" gracePeriod=15 Aug 05 17:27:04.425303 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.425272 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:27:04.531766 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531687 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-config\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.531911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531779 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz4f9\" (UniqueName: \"kubernetes.io/projected/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-kube-api-access-lz4f9\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.531911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531804 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-trusted-ca-bundle\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.531911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531818 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-oauth-serving-cert\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.531911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531841 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-oauth-config\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.531911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531864 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-serving-cert\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.531911 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.531898 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-service-ca\") pod \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\" (UID: \"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068\") " Aug 05 17:27:04.532243 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.532133 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-config" (OuterVolumeSpecName: "console-config") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:27:04.532323 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.532292 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:27:04.532323 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.532308 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:27:04.532515 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.532489 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:27:04.533826 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.533803 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-kube-api-access-lz4f9" (OuterVolumeSpecName: "kube-api-access-lz4f9") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "kube-api-access-lz4f9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:27:04.533914 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.533853 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:27:04.533973 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.533956 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" (UID: "c4bc1eba-0c20-4f6b-b3bd-e0e335dce068"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:27:04.633168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633130 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:04.633168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633162 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-config\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:04.633168 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633174 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz4f9\" (UniqueName: \"kubernetes.io/projected/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-kube-api-access-lz4f9\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:04.633390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633186 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-trusted-ca-bundle\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:04.633390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633198 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-oauth-serving-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:04.633390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633209 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-oauth-config\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:04.633390 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:04.633219 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068-console-serving-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:27:05.285992 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.285916 2575 generic.go:356] "Generic (PLEG): container finished" podID="c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" containerID="d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835" exitCode=0 Aug 05 17:27:05.285992 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.285960 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bddf6b9f7-nbxrr" Aug 05 17:27:05.286413 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.285992 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bddf6b9f7-nbxrr" event={"ID":"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068","Type":"ContainerDied","Data":"d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835"} Aug 05 17:27:05.286413 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.286042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bddf6b9f7-nbxrr" event={"ID":"c4bc1eba-0c20-4f6b-b3bd-e0e335dce068","Type":"ContainerDied","Data":"6d547221471c969d0a75432817528bc078e1a91cdd03ca22064485c9aa16f8b8"} Aug 05 17:27:05.286413 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.286083 2575 scope.go:117] "RemoveContainer" containerID="d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835" Aug 05 17:27:05.294510 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.294492 2575 scope.go:117] "RemoveContainer" containerID="d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835" Aug 05 17:27:05.294738 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:27:05.294721 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835\": container with ID starting with d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835 not found: ID does not exist" containerID="d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835" Aug 05 17:27:05.294789 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.294744 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835"} err="failed to get container status \"d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835\": rpc error: code = NotFound desc = could not find container \"d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835\": container with ID starting with d4be8b28391beed8d9dddbb90aa787eff929c654bc6f068ddbe0ac882eb65835 not found: ID does not exist" Aug 05 17:27:05.307654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.307631 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bddf6b9f7-nbxrr"] Aug 05 17:27:05.311080 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:05.311036 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bddf6b9f7-nbxrr"] Aug 05 17:27:07.026268 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:07.026236 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" path="/var/lib/kubelet/pods/c4bc1eba-0c20-4f6b-b3bd-e0e335dce068/volumes" Aug 05 17:27:09.659694 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.659653 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9"] Aug 05 17:27:09.660225 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660207 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="util" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660229 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="util" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660239 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="util" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660246 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="util" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660256 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="pull" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660264 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="pull" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660277 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="extract" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660285 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="extract" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660295 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="pull" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660303 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="pull" Aug 05 17:27:09.660313 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660315 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="pull" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660323 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="pull" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660331 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660340 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660353 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" containerName="console" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660362 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" containerName="console" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660387 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="util" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660395 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="util" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660404 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="util" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660411 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="util" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660423 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="pull" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660431 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="pull" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660440 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660448 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660457 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660464 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660552 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="141b72a3-e18e-49e9-8ca1-f97497aaa1b3" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660566 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="10ce79f0-08f9-4c3a-a51d-e7a36a79bb98" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660575 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4bc1eba-0c20-4f6b-b3bd-e0e335dce068" containerName="console" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660583 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="499ffb1c-806c-4dd1-b37b-a5b97b74695f" containerName="extract" Aug 05 17:27:09.660749 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.660591 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef5d311-3675-459b-a82d-8690d618bab2" containerName="extract" Aug 05 17:27:09.665510 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.665487 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:09.668326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.668305 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Aug 05 17:27:09.668326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.668326 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 05 17:27:09.668567 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.668536 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 05 17:27:09.668909 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.668894 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9nkrq\"" Aug 05 17:27:09.683279 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.683258 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9"] Aug 05 17:27:09.781741 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.781710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45wrj\" (UniqueName: \"kubernetes.io/projected/387b5967-f81f-4684-9172-b43e9ade410d-kube-api-access-45wrj\") pod \"dns-operator-controller-manager-5985564776-2tnj9\" (UID: \"387b5967-f81f-4684-9172-b43e9ade410d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:09.883179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.883149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wrj\" (UniqueName: \"kubernetes.io/projected/387b5967-f81f-4684-9172-b43e9ade410d-kube-api-access-45wrj\") pod \"dns-operator-controller-manager-5985564776-2tnj9\" (UID: \"387b5967-f81f-4684-9172-b43e9ade410d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:09.893359 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.893334 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45wrj\" (UniqueName: \"kubernetes.io/projected/387b5967-f81f-4684-9172-b43e9ade410d-kube-api-access-45wrj\") pod \"dns-operator-controller-manager-5985564776-2tnj9\" (UID: \"387b5967-f81f-4684-9172-b43e9ade410d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:09.975720 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:09.975649 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:10.108401 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:10.108374 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9"] Aug 05 17:27:10.109638 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:27:10.109608 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387b5967_f81f_4684_9172_b43e9ade410d.slice/crio-006140ad5a002ecfbabb58a4c644fa35332bea32bd1087642d16b42774ba0a72 WatchSource:0}: Error finding container 006140ad5a002ecfbabb58a4c644fa35332bea32bd1087642d16b42774ba0a72: Status 404 returned error can't find the container with id 006140ad5a002ecfbabb58a4c644fa35332bea32bd1087642d16b42774ba0a72 Aug 05 17:27:10.315144 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:10.315108 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" event={"ID":"387b5967-f81f-4684-9172-b43e9ade410d","Type":"ContainerStarted","Data":"006140ad5a002ecfbabb58a4c644fa35332bea32bd1087642d16b42774ba0a72"} Aug 05 17:27:13.341541 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:13.341496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" event={"ID":"387b5967-f81f-4684-9172-b43e9ade410d","Type":"ContainerStarted","Data":"b1ed7d646277e565b8db7a2e15f1d6f46a50a3fdbfef2d00c829c2a1c31c3818"} Aug 05 17:27:13.341944 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:13.341669 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:13.361043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:13.360994 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" podStartSLOduration=1.750749901 podStartE2EDuration="4.360980713s" totalImagesPullingTime="2.610230812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:13.359340387 +0000 UTC m=+470.760569210" watchObservedRunningTime="2026-08-05 17:27:13.360980713 +0000 UTC m=+470.762209527" Aug 05 17:27:14.202756 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.202726 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8"] Aug 05 17:27:14.206306 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.206288 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.210274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.210253 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-skc6r\"" Aug 05 17:27:14.210274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.210269 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Aug 05 17:27:14.210415 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.210255 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Aug 05 17:27:14.216474 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.216454 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gspmv\" (UniqueName: \"kubernetes.io/projected/60f7b3b1-811f-4bd0-b4db-61d655f014ed-kube-api-access-gspmv\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.216568 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.216505 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60f7b3b1-811f-4bd0-b4db-61d655f014ed-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.216568 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.216540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60f7b3b1-811f-4bd0-b4db-61d655f014ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.233646 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.233621 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8"] Aug 05 17:27:14.317047 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.317012 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gspmv\" (UniqueName: \"kubernetes.io/projected/60f7b3b1-811f-4bd0-b4db-61d655f014ed-kube-api-access-gspmv\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.317234 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.317117 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60f7b3b1-811f-4bd0-b4db-61d655f014ed-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.317234 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.317154 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60f7b3b1-811f-4bd0-b4db-61d655f014ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.317316 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:27:14.317254 2575 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Aug 05 17:27:14.317351 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:27:14.317319 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60f7b3b1-811f-4bd0-b4db-61d655f014ed-plugin-serving-cert podName:60f7b3b1-811f-4bd0-b4db-61d655f014ed nodeName:}" failed. No retries permitted until 2026-08-05 17:27:14.817299836 +0000 UTC m=+472.218528628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/60f7b3b1-811f-4bd0-b4db-61d655f014ed-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-w7rv8" (UID: "60f7b3b1-811f-4bd0-b4db-61d655f014ed") : secret "plugin-serving-cert" not found Aug 05 17:27:14.317821 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.317800 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/60f7b3b1-811f-4bd0-b4db-61d655f014ed-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.337624 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.337594 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gspmv\" (UniqueName: \"kubernetes.io/projected/60f7b3b1-811f-4bd0-b4db-61d655f014ed-kube-api-access-gspmv\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.822724 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.822686 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60f7b3b1-811f-4bd0-b4db-61d655f014ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:14.825082 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:14.825032 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/60f7b3b1-811f-4bd0-b4db-61d655f014ed-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-w7rv8\" (UID: \"60f7b3b1-811f-4bd0-b4db-61d655f014ed\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:15.115448 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:15.115357 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" Aug 05 17:27:15.251604 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:15.251579 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8"] Aug 05 17:27:15.253882 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:27:15.253853 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60f7b3b1_811f_4bd0_b4db_61d655f014ed.slice/crio-10613de24fc5286ef7f4c8b816a6cf2ad0fd3c4462857f3e04aba81604331278 WatchSource:0}: Error finding container 10613de24fc5286ef7f4c8b816a6cf2ad0fd3c4462857f3e04aba81604331278: Status 404 returned error can't find the container with id 10613de24fc5286ef7f4c8b816a6cf2ad0fd3c4462857f3e04aba81604331278 Aug 05 17:27:15.353152 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:15.353121 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" event={"ID":"60f7b3b1-811f-4bd0-b4db-61d655f014ed","Type":"ContainerStarted","Data":"10613de24fc5286ef7f4c8b816a6cf2ad0fd3c4462857f3e04aba81604331278"} Aug 05 17:27:20.385613 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:20.385572 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" event={"ID":"60f7b3b1-811f-4bd0-b4db-61d655f014ed","Type":"ContainerStarted","Data":"749e57dc76eeff17778d010c1024d521bc55f926a8927a956a472199da384c5d"} Aug 05 17:27:20.404070 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:20.404011 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-w7rv8" podStartSLOduration=1.7513216090000001 podStartE2EDuration="6.403996982s" totalImagesPullingTime="4.652675373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:20.402407647 +0000 UTC m=+477.803636461" watchObservedRunningTime="2026-08-05 17:27:20.403996982 +0000 UTC m=+477.805225793" Aug 05 17:27:24.349724 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:24.349693 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-2tnj9" Aug 05 17:27:57.635227 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.635192 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:27:57.656834 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.656808 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:27:57.656983 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.656896 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.659300 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.659278 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 05 17:27:57.665234 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.665216 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:27:57.797264 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.797228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/76f17861-853b-4a7f-8e21-267baf2dd452-config-file\") pod \"limitador-limitador-85d9db4d5d-sq7qh\" (UID: \"76f17861-853b-4a7f-8e21-267baf2dd452\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.797419 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.797341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z7lc\" (UniqueName: \"kubernetes.io/projected/76f17861-853b-4a7f-8e21-267baf2dd452-kube-api-access-5z7lc\") pod \"limitador-limitador-85d9db4d5d-sq7qh\" (UID: \"76f17861-853b-4a7f-8e21-267baf2dd452\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.898536 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.898447 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/76f17861-853b-4a7f-8e21-267baf2dd452-config-file\") pod \"limitador-limitador-85d9db4d5d-sq7qh\" (UID: \"76f17861-853b-4a7f-8e21-267baf2dd452\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.898698 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.898559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z7lc\" (UniqueName: \"kubernetes.io/projected/76f17861-853b-4a7f-8e21-267baf2dd452-kube-api-access-5z7lc\") pod \"limitador-limitador-85d9db4d5d-sq7qh\" (UID: \"76f17861-853b-4a7f-8e21-267baf2dd452\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.899099 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.899079 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/76f17861-853b-4a7f-8e21-267baf2dd452-config-file\") pod \"limitador-limitador-85d9db4d5d-sq7qh\" (UID: \"76f17861-853b-4a7f-8e21-267baf2dd452\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.906841 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.906817 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z7lc\" (UniqueName: \"kubernetes.io/projected/76f17861-853b-4a7f-8e21-267baf2dd452-kube-api-access-5z7lc\") pod \"limitador-limitador-85d9db4d5d-sq7qh\" (UID: \"76f17861-853b-4a7f-8e21-267baf2dd452\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:57.967697 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:57.967666 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:58.095587 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.095554 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:27:58.097806 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:27:58.097780 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f17861_853b_4a7f_8e21_267baf2dd452.slice/crio-f30cd5732e1ba68aac1f024957881d1e604aef50d17da791fc80c8c044e57121 WatchSource:0}: Error finding container f30cd5732e1ba68aac1f024957881d1e604aef50d17da791fc80c8c044e57121: Status 404 returned error can't find the container with id f30cd5732e1ba68aac1f024957881d1e604aef50d17da791fc80c8c044e57121 Aug 05 17:27:58.182380 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.182307 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xnc7k"] Aug 05 17:27:58.187110 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.187043 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:27:58.189255 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.189229 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8kzqt\"" Aug 05 17:27:58.192993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.192970 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xnc7k"] Aug 05 17:27:58.302201 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.302166 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spltf\" (UniqueName: \"kubernetes.io/projected/93af1a14-c29d-4ae2-9f2f-adb75711e6a6-kube-api-access-spltf\") pod \"authorino-c548b5d7d-xnc7k\" (UID: \"93af1a14-c29d-4ae2-9f2f-adb75711e6a6\") " pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:27:58.403432 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.403382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spltf\" (UniqueName: \"kubernetes.io/projected/93af1a14-c29d-4ae2-9f2f-adb75711e6a6-kube-api-access-spltf\") pod \"authorino-c548b5d7d-xnc7k\" (UID: \"93af1a14-c29d-4ae2-9f2f-adb75711e6a6\") " pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:27:58.412380 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.412355 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spltf\" (UniqueName: \"kubernetes.io/projected/93af1a14-c29d-4ae2-9f2f-adb75711e6a6-kube-api-access-spltf\") pod \"authorino-c548b5d7d-xnc7k\" (UID: \"93af1a14-c29d-4ae2-9f2f-adb75711e6a6\") " pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:27:58.497900 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.497822 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:27:58.593908 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.593874 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" event={"ID":"76f17861-853b-4a7f-8e21-267baf2dd452","Type":"ContainerStarted","Data":"f30cd5732e1ba68aac1f024957881d1e604aef50d17da791fc80c8c044e57121"} Aug 05 17:27:58.643613 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:58.643582 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xnc7k"] Aug 05 17:27:58.646468 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:27:58.646436 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93af1a14_c29d_4ae2_9f2f_adb75711e6a6.slice/crio-2f6c289ae3006a08f51925ec6e18843a15a42be8310685c29d96d342907f4b78 WatchSource:0}: Error finding container 2f6c289ae3006a08f51925ec6e18843a15a42be8310685c29d96d342907f4b78: Status 404 returned error can't find the container with id 2f6c289ae3006a08f51925ec6e18843a15a42be8310685c29d96d342907f4b78 Aug 05 17:27:59.601253 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:59.601209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" event={"ID":"76f17861-853b-4a7f-8e21-267baf2dd452","Type":"ContainerStarted","Data":"c59358fd43aa71e6beb94d657d85fabd4729d5499574fa819fbd2787ae14e500"} Aug 05 17:27:59.601450 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:59.601395 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:27:59.602392 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:59.602365 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" event={"ID":"93af1a14-c29d-4ae2-9f2f-adb75711e6a6","Type":"ContainerStarted","Data":"2f6c289ae3006a08f51925ec6e18843a15a42be8310685c29d96d342907f4b78"} Aug 05 17:27:59.619246 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:27:59.619196 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" podStartSLOduration=1.731716466 podStartE2EDuration="2.619181308s" totalImagesPullingTime="887.464842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:59.61713313 +0000 UTC m=+517.018361945" watchObservedRunningTime="2026-08-05 17:27:59.619181308 +0000 UTC m=+517.020410121" Aug 05 17:28:01.614762 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:01.614724 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" event={"ID":"93af1a14-c29d-4ae2-9f2f-adb75711e6a6","Type":"ContainerStarted","Data":"e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76"} Aug 05 17:28:01.630569 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:01.630519 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" podStartSLOduration=1.5245739820000002 podStartE2EDuration="3.630502906s" totalImagesPullingTime="2.105928924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:01.628697387 +0000 UTC m=+519.029926217" watchObservedRunningTime="2026-08-05 17:28:01.630502906 +0000 UTC m=+519.031731749" Aug 05 17:28:10.610000 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:10.609969 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh" Aug 05 17:28:26.692427 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.692349 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-mtdsx"] Aug 05 17:28:26.695776 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.695758 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:26.705080 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.705035 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-mtdsx"] Aug 05 17:28:26.745744 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.745716 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-mtdsx"] Aug 05 17:28:26.745988 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:28:26.745968 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lctfv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" podUID="ac8f46d6-bf60-481f-af0b-50f34eeedd3c" Aug 05 17:28:26.757797 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.757768 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/ac8f46d6-bf60-481f-af0b-50f34eeedd3c-kube-api-access-lctfv\") pod \"authorino-5dc7ccb5f-mtdsx\" (UID: \"ac8f46d6-bf60-481f-af0b-50f34eeedd3c\") " pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:26.858916 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.858882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/ac8f46d6-bf60-481f-af0b-50f34eeedd3c-kube-api-access-lctfv\") pod \"authorino-5dc7ccb5f-mtdsx\" (UID: \"ac8f46d6-bf60-481f-af0b-50f34eeedd3c\") " pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:26.866724 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.866698 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/ac8f46d6-bf60-481f-af0b-50f34eeedd3c-kube-api-access-lctfv\") pod \"authorino-5dc7ccb5f-mtdsx\" (UID: \"ac8f46d6-bf60-481f-af0b-50f34eeedd3c\") " pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:26.999463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:26.999387 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cdfd759d5-5x6jq"] Aug 05 17:28:27.002699 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.002683 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.004893 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.004871 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Aug 05 17:28:27.009976 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.009953 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cdfd759d5-5x6jq"] Aug 05 17:28:27.060281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.060213 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-tls-cert\") pod \"authorino-cdfd759d5-5x6jq\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.060423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.060333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wvx6\" (UniqueName: \"kubernetes.io/projected/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-kube-api-access-9wvx6\") pod \"authorino-cdfd759d5-5x6jq\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.161195 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.161159 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wvx6\" (UniqueName: \"kubernetes.io/projected/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-kube-api-access-9wvx6\") pod \"authorino-cdfd759d5-5x6jq\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.161343 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.161243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-tls-cert\") pod \"authorino-cdfd759d5-5x6jq\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.163607 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.163577 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-tls-cert\") pod \"authorino-cdfd759d5-5x6jq\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.168923 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.168897 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wvx6\" (UniqueName: \"kubernetes.io/projected/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-kube-api-access-9wvx6\") pod \"authorino-cdfd759d5-5x6jq\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.313797 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.313771 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:28:27.439929 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.439905 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cdfd759d5-5x6jq"] Aug 05 17:28:27.441738 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:28:27.441712 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fbe8b8e_04a8_4ebf_a71e_34d07428057c.slice/crio-abb964e51231ca52183b5fb01316622b8db917a55bb2ea2549c2fec8c292c5e7 WatchSource:0}: Error finding container abb964e51231ca52183b5fb01316622b8db917a55bb2ea2549c2fec8c292c5e7: Status 404 returned error can't find the container with id abb964e51231ca52183b5fb01316622b8db917a55bb2ea2549c2fec8c292c5e7 Aug 05 17:28:27.750232 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.750143 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" event={"ID":"0fbe8b8e-04a8-4ebf-a71e-34d07428057c","Type":"ContainerStarted","Data":"abb964e51231ca52183b5fb01316622b8db917a55bb2ea2549c2fec8c292c5e7"} Aug 05 17:28:27.750232 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.750200 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:27.758197 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.758176 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:27.868018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.867985 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/ac8f46d6-bf60-481f-af0b-50f34eeedd3c-kube-api-access-lctfv\") pod \"ac8f46d6-bf60-481f-af0b-50f34eeedd3c\" (UID: \"ac8f46d6-bf60-481f-af0b-50f34eeedd3c\") " Aug 05 17:28:27.870088 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.870065 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac8f46d6-bf60-481f-af0b-50f34eeedd3c-kube-api-access-lctfv" (OuterVolumeSpecName: "kube-api-access-lctfv") pod "ac8f46d6-bf60-481f-af0b-50f34eeedd3c" (UID: "ac8f46d6-bf60-481f-af0b-50f34eeedd3c"). InnerVolumeSpecName "kube-api-access-lctfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:28:27.968909 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:27.968873 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lctfv\" (UniqueName: \"kubernetes.io/projected/ac8f46d6-bf60-481f-af0b-50f34eeedd3c-kube-api-access-lctfv\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:28:28.755493 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:28.755459 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-mtdsx" Aug 05 17:28:28.784453 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:28.784421 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-mtdsx"] Aug 05 17:28:28.790316 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:28.790289 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-mtdsx"] Aug 05 17:28:29.026406 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:29.026329 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac8f46d6-bf60-481f-af0b-50f34eeedd3c" path="/var/lib/kubelet/pods/ac8f46d6-bf60-481f-af0b-50f34eeedd3c/volumes" Aug 05 17:28:30.767178 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:30.767084 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" event={"ID":"0fbe8b8e-04a8-4ebf-a71e-34d07428057c","Type":"ContainerStarted","Data":"7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad"} Aug 05 17:28:30.783423 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:30.783370 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" podStartSLOduration=1.900233837 podStartE2EDuration="4.783354631s" totalImagesPullingTime="2.883120794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:28:30.781788825 +0000 UTC m=+548.183017638" watchObservedRunningTime="2026-08-05 17:28:30.783354631 +0000 UTC m=+548.184583444" Aug 05 17:28:30.811429 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:30.811394 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xnc7k"] Aug 05 17:28:30.811627 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:30.811599 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" podUID="93af1a14-c29d-4ae2-9f2f-adb75711e6a6" containerName="authorino" containerID="cri-o://e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76" gracePeriod=30 Aug 05 17:28:31.049838 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.049810 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:28:31.198973 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.198941 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spltf\" (UniqueName: \"kubernetes.io/projected/93af1a14-c29d-4ae2-9f2f-adb75711e6a6-kube-api-access-spltf\") pod \"93af1a14-c29d-4ae2-9f2f-adb75711e6a6\" (UID: \"93af1a14-c29d-4ae2-9f2f-adb75711e6a6\") " Aug 05 17:28:31.201066 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.201028 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93af1a14-c29d-4ae2-9f2f-adb75711e6a6-kube-api-access-spltf" (OuterVolumeSpecName: "kube-api-access-spltf") pod "93af1a14-c29d-4ae2-9f2f-adb75711e6a6" (UID: "93af1a14-c29d-4ae2-9f2f-adb75711e6a6"). InnerVolumeSpecName "kube-api-access-spltf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:28:31.300126 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.300102 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spltf\" (UniqueName: \"kubernetes.io/projected/93af1a14-c29d-4ae2-9f2f-adb75711e6a6-kube-api-access-spltf\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:28:31.774207 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.774121 2575 generic.go:356] "Generic (PLEG): container finished" podID="93af1a14-c29d-4ae2-9f2f-adb75711e6a6" containerID="e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76" exitCode=0 Aug 05 17:28:31.774207 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.774161 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" Aug 05 17:28:31.774687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.774204 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" event={"ID":"93af1a14-c29d-4ae2-9f2f-adb75711e6a6","Type":"ContainerDied","Data":"e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76"} Aug 05 17:28:31.774687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.774239 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xnc7k" event={"ID":"93af1a14-c29d-4ae2-9f2f-adb75711e6a6","Type":"ContainerDied","Data":"2f6c289ae3006a08f51925ec6e18843a15a42be8310685c29d96d342907f4b78"} Aug 05 17:28:31.774687 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.774255 2575 scope.go:117] "RemoveContainer" containerID="e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76" Aug 05 17:28:31.783822 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.783804 2575 scope.go:117] "RemoveContainer" containerID="e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76" Aug 05 17:28:31.784086 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:28:31.784065 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76\": container with ID starting with e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76 not found: ID does not exist" containerID="e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76" Aug 05 17:28:31.784142 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.784095 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76"} err="failed to get container status \"e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76\": rpc error: code = NotFound desc = could not find container \"e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76\": container with ID starting with e07d1637a15cbcf516526fba085e5511f977427a3923e51c6855e06eb2775e76 not found: ID does not exist" Aug 05 17:28:31.796143 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.796121 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xnc7k"] Aug 05 17:28:31.800068 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:31.800031 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xnc7k"] Aug 05 17:28:33.026881 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:28:33.026842 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93af1a14-c29d-4ae2-9f2f-adb75711e6a6" path="/var/lib/kubelet/pods/93af1a14-c29d-4ae2-9f2f-adb75711e6a6/volumes" Aug 05 17:29:13.874833 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:13.874793 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:29:22.984665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:22.984624 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:29:22.986024 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:22.986006 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:29:22.990238 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:22.990208 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:29:22.991774 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:22.991751 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:29:36.199546 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:36.199506 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:29:44.970543 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:44.970501 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:29:51.065607 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:29:51.065572 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:30:05.980636 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:30:05.980585 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:30:13.771679 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:30:13.771640 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:30:23.567561 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:30:23.567516 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:31:33.943235 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:33.943149 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c77fb47d4-cskr9"] Aug 05 17:31:33.943702 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:33.943603 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93af1a14-c29d-4ae2-9f2f-adb75711e6a6" containerName="authorino" Aug 05 17:31:33.943702 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:33.943616 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="93af1a14-c29d-4ae2-9f2f-adb75711e6a6" containerName="authorino" Aug 05 17:31:33.943702 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:33.943679 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="93af1a14-c29d-4ae2-9f2f-adb75711e6a6" containerName="authorino" Aug 05 17:31:33.946711 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:33.946691 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:33.954726 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:33.954701 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c77fb47d4-cskr9"] Aug 05 17:31:34.037620 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.037580 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxt9g\" (UniqueName: \"kubernetes.io/projected/fb6cdc6d-3d42-4539-bfaa-f9268b517fef-kube-api-access-mxt9g\") pod \"authorino-6c77fb47d4-cskr9\" (UID: \"fb6cdc6d-3d42-4539-bfaa-f9268b517fef\") " pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.037801 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.037630 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fb6cdc6d-3d42-4539-bfaa-f9268b517fef-tls-cert\") pod \"authorino-6c77fb47d4-cskr9\" (UID: \"fb6cdc6d-3d42-4539-bfaa-f9268b517fef\") " pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.138657 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.138624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxt9g\" (UniqueName: \"kubernetes.io/projected/fb6cdc6d-3d42-4539-bfaa-f9268b517fef-kube-api-access-mxt9g\") pod \"authorino-6c77fb47d4-cskr9\" (UID: \"fb6cdc6d-3d42-4539-bfaa-f9268b517fef\") " pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.138848 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.138668 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fb6cdc6d-3d42-4539-bfaa-f9268b517fef-tls-cert\") pod \"authorino-6c77fb47d4-cskr9\" (UID: \"fb6cdc6d-3d42-4539-bfaa-f9268b517fef\") " pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.141154 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.141130 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fb6cdc6d-3d42-4539-bfaa-f9268b517fef-tls-cert\") pod \"authorino-6c77fb47d4-cskr9\" (UID: \"fb6cdc6d-3d42-4539-bfaa-f9268b517fef\") " pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.147606 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.147576 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxt9g\" (UniqueName: \"kubernetes.io/projected/fb6cdc6d-3d42-4539-bfaa-f9268b517fef-kube-api-access-mxt9g\") pod \"authorino-6c77fb47d4-cskr9\" (UID: \"fb6cdc6d-3d42-4539-bfaa-f9268b517fef\") " pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.257220 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.257125 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c77fb47d4-cskr9" Aug 05 17:31:34.390379 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.390356 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c77fb47d4-cskr9"] Aug 05 17:31:34.392156 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:31:34.392120 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb6cdc6d_3d42_4539_bfaa_f9268b517fef.slice/crio-199c988fcd9e9df0ab04d839680239be0ab94dcd5b10c0a515cb66f3a7195a59 WatchSource:0}: Error finding container 199c988fcd9e9df0ab04d839680239be0ab94dcd5b10c0a515cb66f3a7195a59: Status 404 returned error can't find the container with id 199c988fcd9e9df0ab04d839680239be0ab94dcd5b10c0a515cb66f3a7195a59 Aug 05 17:31:34.393436 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.393415 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 17:31:34.750781 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:34.750748 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c77fb47d4-cskr9" event={"ID":"fb6cdc6d-3d42-4539-bfaa-f9268b517fef","Type":"ContainerStarted","Data":"199c988fcd9e9df0ab04d839680239be0ab94dcd5b10c0a515cb66f3a7195a59"} Aug 05 17:31:35.757758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:35.757717 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c77fb47d4-cskr9" event={"ID":"fb6cdc6d-3d42-4539-bfaa-f9268b517fef","Type":"ContainerStarted","Data":"40aba33a89a7225af7cb807400ab68de9be3a943e558e6232ef7455d06a47a3b"} Aug 05 17:31:35.781792 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:35.781732 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c77fb47d4-cskr9" podStartSLOduration=2.428359432 podStartE2EDuration="2.781713216s" totalImagesPullingTime="353.353784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:31:35.780927386 +0000 UTC m=+733.182156201" watchObservedRunningTime="2026-08-05 17:31:35.781713216 +0000 UTC m=+733.182942034" Aug 05 17:31:35.810785 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:35.810746 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cdfd759d5-5x6jq"] Aug 05 17:31:35.810987 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:35.810964 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" podUID="0fbe8b8e-04a8-4ebf-a71e-34d07428057c" containerName="authorino" containerID="cri-o://7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad" gracePeriod=30 Aug 05 17:31:36.055332 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.055308 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:31:36.156860 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.156824 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wvx6\" (UniqueName: \"kubernetes.io/projected/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-kube-api-access-9wvx6\") pod \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " Aug 05 17:31:36.157039 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.156932 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-tls-cert\") pod \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\" (UID: \"0fbe8b8e-04a8-4ebf-a71e-34d07428057c\") " Aug 05 17:31:36.159012 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.158974 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-kube-api-access-9wvx6" (OuterVolumeSpecName: "kube-api-access-9wvx6") pod "0fbe8b8e-04a8-4ebf-a71e-34d07428057c" (UID: "0fbe8b8e-04a8-4ebf-a71e-34d07428057c"). InnerVolumeSpecName "kube-api-access-9wvx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:31:36.167763 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.167728 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0fbe8b8e-04a8-4ebf-a71e-34d07428057c" (UID: "0fbe8b8e-04a8-4ebf-a71e-34d07428057c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 17:31:36.258497 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.258455 2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-tls-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:31:36.258497 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.258496 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wvx6\" (UniqueName: \"kubernetes.io/projected/0fbe8b8e-04a8-4ebf-a71e-34d07428057c-kube-api-access-9wvx6\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:31:36.764593 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.764552 2575 generic.go:356] "Generic (PLEG): container finished" podID="0fbe8b8e-04a8-4ebf-a71e-34d07428057c" containerID="7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad" exitCode=0 Aug 05 17:31:36.765018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.764600 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" Aug 05 17:31:36.765018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.764639 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" event={"ID":"0fbe8b8e-04a8-4ebf-a71e-34d07428057c","Type":"ContainerDied","Data":"7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad"} Aug 05 17:31:36.765018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.764686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cdfd759d5-5x6jq" event={"ID":"0fbe8b8e-04a8-4ebf-a71e-34d07428057c","Type":"ContainerDied","Data":"abb964e51231ca52183b5fb01316622b8db917a55bb2ea2549c2fec8c292c5e7"} Aug 05 17:31:36.765018 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.764705 2575 scope.go:117] "RemoveContainer" containerID="7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad" Aug 05 17:31:36.774234 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.774214 2575 scope.go:117] "RemoveContainer" containerID="7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad" Aug 05 17:31:36.774493 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:31:36.774475 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad\": container with ID starting with 7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad not found: ID does not exist" containerID="7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad" Aug 05 17:31:36.774550 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.774501 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad"} err="failed to get container status \"7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad\": rpc error: code = NotFound desc = could not find container \"7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad\": container with ID starting with 7cf77508ca0d9b6a2c1b3b84d655a727868015fd07b2f387e51857ab682a57ad not found: ID does not exist" Aug 05 17:31:36.799465 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.799436 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cdfd759d5-5x6jq"] Aug 05 17:31:36.812896 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:36.812866 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cdfd759d5-5x6jq"] Aug 05 17:31:37.027577 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:31:37.027501 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fbe8b8e-04a8-4ebf-a71e-34d07428057c" path="/var/lib/kubelet/pods/0fbe8b8e-04a8-4ebf-a71e-34d07428057c/volumes" Aug 05 17:32:04.466173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:32:04.466130 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:32:14.674905 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:32:14.674868 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:32:27.064120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:32:27.064086 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:34:07.391376 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:34:07.391338 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:34:17.181781 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:34:17.181697 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:34:23.023788 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:34:23.023753 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:34:23.024278 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:34:23.023756 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:34:23.029903 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:34:23.029878 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:34:23.030135 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:34:23.030125 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:35:16.672596 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:35:16.672557 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:35:24.184896 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:35:24.184855 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:35:39.681242 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:35:39.681204 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:35:45.469591 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:35:45.469514 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:35:55.293735 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:35:55.293696 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:02.485604 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:02.485570 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:15.879296 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:15.879259 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:21.092884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:21.092852 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:26.178922 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:26.178876 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:33.116443 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:33.116404 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:38.524312 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:38.524270 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:36:51.281722 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:36:51.281689 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:37:01.972178 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:37:01.972140 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:37:26.274140 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:37:26.274040 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:37:36.675701 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:37:36.675664 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:37:42.373496 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:37:42.373459 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:37:49.768166 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:37:49.768129 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:37:57.124786 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:37:57.124741 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:02.869356 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:02.869325 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:08.567100 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:08.567045 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:15.861561 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:15.861524 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:27.510697 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:27.510656 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:35.446298 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:35.446264 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:45.963515 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:45.963437 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:50.875439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:50.875404 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:38:56.977204 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:38:56.977166 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:39:06.164876 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:06.164837 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:39:11.772945 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:11.772909 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:39:23.067274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:23.067245 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:39:23.071035 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:23.071011 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:39:23.073116 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:23.073089 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:39:23.074556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:23.074533 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:39:23.077520 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:39:23.077504 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:41:37.859654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:41:37.859621 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:41:42.775666 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:41:42.775627 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:42:06.876344 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:42:06.876266 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:08.870995 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:08.870952 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:18.265659 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:18.265577 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:23.568309 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:23.568273 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:28.573556 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:28.573512 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:39.069152 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:39.069118 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:45.761236 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:45.761202 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:54.402499 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:54.402463 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:43:59.564668 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:43:59.564630 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:44:09.570120 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:09.570073 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:44:16.068564 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:16.068527 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:44:23.105759 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:23.105734 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:44:23.111255 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:23.111233 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:44:23.111976 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:23.111957 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:44:23.123559 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:23.123542 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:44:25.468512 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:25.468474 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:44:47.864381 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:47.864300 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:44:54.366862 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:44:54.366822 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:45:16.572954 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:45:16.572914 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:45:26.180403 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:45:26.180361 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:02.965754 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:02.965720 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:13.870134 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:13.870099 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:18.862577 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:18.862501 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:27.560917 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:27.560876 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:34.563413 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:34.563373 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:40.571673 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:40.571633 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:46.871165 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:46.871130 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:53.961993 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:53.961961 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:46:58.463849 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:46:58.463817 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:08.577292 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:08.577253 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:17.266068 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:17.266015 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:25.260621 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:25.260588 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:31.382714 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:31.382677 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:38.064465 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:38.064432 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:49.875167 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:49.875080 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:47:57.063956 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:47:57.063924 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:04.966638 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:04.966602 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:08.867267 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:08.867237 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:14.970199 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:14.970158 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:24.668170 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:24.668134 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:30.770385 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:30.770337 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:41.960904 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:41.960868 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:48.368614 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:48.368579 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:48:58.863342 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:48:58.863294 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:49:13.968821 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:13.968780 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:49:19.875670 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:19.875588 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:49:23.154321 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:23.154292 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:49:23.160287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:23.160265 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:49:23.166344 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:23.166325 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:49:23.172457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:23.172441 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:49:26.235795 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:26.235757 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:49:38.065439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:38.065404 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:49:46.368725 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:46.368686 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:49:51.870457 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:49:51.870419 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:50:05.469890 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:05.469857 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:50:27.121836 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.118708 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk"] Aug 05 17:50:27.121836 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.119591 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fbe8b8e-04a8-4ebf-a71e-34d07428057c" containerName="authorino" Aug 05 17:50:27.121836 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.119611 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fbe8b8e-04a8-4ebf-a71e-34d07428057c" containerName="authorino" Aug 05 17:50:27.121836 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.119822 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fbe8b8e-04a8-4ebf-a71e-34d07428057c" containerName="authorino" Aug 05 17:50:27.124153 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.124119 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:27.126463 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.126435 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 17:50:27.126625 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.126453 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 17:50:27.126736 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.126679 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk"] Aug 05 17:50:27.144935 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.144910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c519e93-78c4-4a98-8f84-9555880ef55c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk\" (UID: \"0c519e93-78c4-4a98-8f84-9555880ef55c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:27.246216 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.246183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c519e93-78c4-4a98-8f84-9555880ef55c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk\" (UID: \"0c519e93-78c4-4a98-8f84-9555880ef55c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:27.246783 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.246758 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c519e93-78c4-4a98-8f84-9555880ef55c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk\" (UID: \"0c519e93-78c4-4a98-8f84-9555880ef55c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:27.436233 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.436148 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:27.569170 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.569147 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk"] Aug 05 17:50:27.571068 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:50:27.571029 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c519e93_78c4_4a98_8f84_9555880ef55c.slice/crio-006d9aade70e16a289a0f6c2637646a0b487b77618cac862da65b2ceac89832a WatchSource:0}: Error finding container 006d9aade70e16a289a0f6c2637646a0b487b77618cac862da65b2ceac89832a: Status 404 returned error can't find the container with id 006d9aade70e16a289a0f6c2637646a0b487b77618cac862da65b2ceac89832a Aug 05 17:50:27.573374 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.573357 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 17:50:27.869821 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:27.869789 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" event={"ID":"0c519e93-78c4-4a98-8f84-9555880ef55c","Type":"ContainerStarted","Data":"006d9aade70e16a289a0f6c2637646a0b487b77618cac862da65b2ceac89832a"} Aug 05 17:50:30.888840 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:30.888806 2575 generic.go:356] "Generic (PLEG): container finished" podID="0c519e93-78c4-4a98-8f84-9555880ef55c" containerID="e505afd504c9b99a632a24f3a3a3cd9dbded3419bc3ddcaf77f30d1eb0b22e23" exitCode=0 Aug 05 17:50:30.889225 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:30.888882 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" event={"ID":"0c519e93-78c4-4a98-8f84-9555880ef55c","Type":"ContainerDied","Data":"e505afd504c9b99a632a24f3a3a3cd9dbded3419bc3ddcaf77f30d1eb0b22e23"} Aug 05 17:50:32.023852 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.023827 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:32.089770 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.089731 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c519e93-78c4-4a98-8f84-9555880ef55c-maas-api-service-ca\") pod \"0c519e93-78c4-4a98-8f84-9555880ef55c\" (UID: \"0c519e93-78c4-4a98-8f84-9555880ef55c\") " Aug 05 17:50:32.090102 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.090080 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c519e93-78c4-4a98-8f84-9555880ef55c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c519e93-78c4-4a98-8f84-9555880ef55c" (UID: "0c519e93-78c4-4a98-8f84-9555880ef55c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:50:32.191178 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.191094 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c519e93-78c4-4a98-8f84-9555880ef55c-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:50:32.901078 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.901028 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" Aug 05 17:50:32.901078 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.901026 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk" event={"ID":"0c519e93-78c4-4a98-8f84-9555880ef55c","Type":"ContainerDied","Data":"006d9aade70e16a289a0f6c2637646a0b487b77618cac862da65b2ceac89832a"} Aug 05 17:50:32.901298 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:32.901117 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="006d9aade70e16a289a0f6c2637646a0b487b77618cac862da65b2ceac89832a" Aug 05 17:50:43.546040 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:43.546001 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk"] Aug 05 17:50:43.548597 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:43.548567 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fbbe1196-f7wxk"] Aug 05 17:50:45.025728 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:45.025654 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c519e93-78c4-4a98-8f84-9555880ef55c" path="/var/lib/kubelet/pods/0c519e93-78c4-4a98-8f84-9555880ef55c/volumes" Aug 05 17:50:46.565215 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:46.565182 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:50:48.513212 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.513170 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l"] Aug 05 17:50:48.513598 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.513545 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c519e93-78c4-4a98-8f84-9555880ef55c" containerName="revoke-keys" Aug 05 17:50:48.513598 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.513556 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c519e93-78c4-4a98-8f84-9555880ef55c" containerName="revoke-keys" Aug 05 17:50:48.513681 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.513630 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c519e93-78c4-4a98-8f84-9555880ef55c" containerName="revoke-keys" Aug 05 17:50:48.517892 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.517875 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.521522 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.521501 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-41ad7fbb-dockercfg-hbxcc\"" Aug 05 17:50:48.521661 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.521535 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-41ad7fbb\"" Aug 05 17:50:48.529607 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.529584 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l"] Aug 05 17:50:48.646942 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.646904 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29kkg\" (UniqueName: \"kubernetes.io/projected/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-kube-api-access-29kkg\") pod \"payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.647136 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.647036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.747669 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.747639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29kkg\" (UniqueName: \"kubernetes.io/projected/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-kube-api-access-29kkg\") pod \"payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.747867 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.747750 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.748505 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.748480 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.757639 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.757616 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29kkg\" (UniqueName: \"kubernetes.io/projected/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-kube-api-access-29kkg\") pod \"payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.828700 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.828667 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:48.960292 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:48.960261 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l"] Aug 05 17:50:48.962261 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:50:48.962235 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed91077_4b42_42ae_a0f7_01ad5b0aa3b6.slice/crio-eba168fdb17a12c34fb084d7acf42ee65edff973e618143c6368144123c28dcf WatchSource:0}: Error finding container eba168fdb17a12c34fb084d7acf42ee65edff973e618143c6368144123c28dcf: Status 404 returned error can't find the container with id eba168fdb17a12c34fb084d7acf42ee65edff973e618143c6368144123c28dcf Aug 05 17:50:49.005508 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:49.005474 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" event={"ID":"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6","Type":"ContainerStarted","Data":"eba168fdb17a12c34fb084d7acf42ee65edff973e618143c6368144123c28dcf"} Aug 05 17:50:51.026690 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:51.026663 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" event={"ID":"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6","Type":"ContainerStarted","Data":"5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677"} Aug 05 17:50:51.027003 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:51.026723 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:50:51.035513 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:51.035461 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" podStartSLOduration=1.051358771 podStartE2EDuration="3.035442077s" totalImagesPullingTime="1.984083306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:50:51.033938507 +0000 UTC m=+1888.435167322" watchObservedRunningTime="2026-08-05 17:50:51.035442077 +0000 UTC m=+1888.436670890" Aug 05 17:50:54.960939 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:50:54.960901 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:51:02.027445 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:02.027411 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:51:12.556971 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.556930 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk"] Aug 05 17:51:12.559440 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.559421 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:12.562180 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.562159 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 17:51:12.562284 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.562159 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 17:51:12.568969 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.568948 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk"] Aug 05 17:51:12.657149 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.657110 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899e3bc-599b-4144-8c00-2c8bdac84706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk\" (UID: \"5899e3bc-599b-4144-8c00-2c8bdac84706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:12.758271 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.758232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899e3bc-599b-4144-8c00-2c8bdac84706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk\" (UID: \"5899e3bc-599b-4144-8c00-2c8bdac84706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:12.758835 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.758817 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899e3bc-599b-4144-8c00-2c8bdac84706-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk\" (UID: \"5899e3bc-599b-4144-8c00-2c8bdac84706\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:12.870214 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.870132 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:12.998350 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:12.998324 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk"] Aug 05 17:51:12.999664 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:51:12.999640 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5899e3bc_599b_4144_8c00_2c8bdac84706.slice/crio-468490e6f9f44094f9ce925af73d411f0aa363f936cb8cd98c5db0ac3dd109f0 WatchSource:0}: Error finding container 468490e6f9f44094f9ce925af73d411f0aa363f936cb8cd98c5db0ac3dd109f0: Status 404 returned error can't find the container with id 468490e6f9f44094f9ce925af73d411f0aa363f936cb8cd98c5db0ac3dd109f0 Aug 05 17:51:13.138606 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:13.138519 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" event={"ID":"5899e3bc-599b-4144-8c00-2c8bdac84706","Type":"ContainerStarted","Data":"468490e6f9f44094f9ce925af73d411f0aa363f936cb8cd98c5db0ac3dd109f0"} Aug 05 17:51:14.146096 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:14.146063 2575 generic.go:356] "Generic (PLEG): container finished" podID="5899e3bc-599b-4144-8c00-2c8bdac84706" containerID="dc0c26ba5aca7c82221d9f2359d4f5b41a985fe15584fb572a43599e19ebc1a8" exitCode=0 Aug 05 17:51:14.146510 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:14.146151 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" event={"ID":"5899e3bc-599b-4144-8c00-2c8bdac84706","Type":"ContainerDied","Data":"dc0c26ba5aca7c82221d9f2359d4f5b41a985fe15584fb572a43599e19ebc1a8"} Aug 05 17:51:15.278710 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:15.278686 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:15.376904 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:15.376877 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899e3bc-599b-4144-8c00-2c8bdac84706-maas-api-service-ca\") pod \"5899e3bc-599b-4144-8c00-2c8bdac84706\" (UID: \"5899e3bc-599b-4144-8c00-2c8bdac84706\") " Aug 05 17:51:15.377232 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:15.377210 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5899e3bc-599b-4144-8c00-2c8bdac84706-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5899e3bc-599b-4144-8c00-2c8bdac84706" (UID: "5899e3bc-599b-4144-8c00-2c8bdac84706"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:51:15.477985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:15.477902 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5899e3bc-599b-4144-8c00-2c8bdac84706-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:51:16.159319 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:16.159289 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" Aug 05 17:51:16.159491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:16.159282 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk" event={"ID":"5899e3bc-599b-4144-8c00-2c8bdac84706","Type":"ContainerDied","Data":"468490e6f9f44094f9ce925af73d411f0aa363f936cb8cd98c5db0ac3dd109f0"} Aug 05 17:51:16.159491 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:16.159422 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="468490e6f9f44094f9ce925af73d411f0aa363f936cb8cd98c5db0ac3dd109f0" Aug 05 17:51:19.099656 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.099621 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l"] Aug 05 17:51:19.100131 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.099860 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" podUID="7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" containerName="payload-processing" containerID="cri-o://5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677" gracePeriod=30 Aug 05 17:51:19.344893 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.344868 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:51:19.358844 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.358750 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk"] Aug 05 17:51:19.361829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.361426 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-41ad7fbb-p74kk"] Aug 05 17:51:19.409029 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.408991 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29kkg\" (UniqueName: \"kubernetes.io/projected/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-kube-api-access-29kkg\") pod \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " Aug 05 17:51:19.409236 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.409145 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-plugins-config-volume\") pod \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\" (UID: \"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6\") " Aug 05 17:51:19.409528 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.409506 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" (UID: "7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:51:19.411272 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.411252 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-kube-api-access-29kkg" (OuterVolumeSpecName: "kube-api-access-29kkg") pod "7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" (UID: "7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6"). InnerVolumeSpecName "kube-api-access-29kkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:51:19.510815 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.510779 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-plugins-config-volume\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:51:19.510815 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:19.510815 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29kkg\" (UniqueName: \"kubernetes.io/projected/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6-kube-api-access-29kkg\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:51:20.182670 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.182637 2575 generic.go:356] "Generic (PLEG): container finished" podID="7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" containerID="5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677" exitCode=0 Aug 05 17:51:20.183130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.182683 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" Aug 05 17:51:20.183130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.182708 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" event={"ID":"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6","Type":"ContainerDied","Data":"5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677"} Aug 05 17:51:20.183130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.182747 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l" event={"ID":"7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6","Type":"ContainerDied","Data":"eba168fdb17a12c34fb084d7acf42ee65edff973e618143c6368144123c28dcf"} Aug 05 17:51:20.183130 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.182767 2575 scope.go:117] "RemoveContainer" containerID="5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677" Aug 05 17:51:20.192198 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.192176 2575 scope.go:117] "RemoveContainer" containerID="5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677" Aug 05 17:51:20.192415 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:51:20.192396 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677\": container with ID starting with 5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677 not found: ID does not exist" containerID="5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677" Aug 05 17:51:20.192475 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.192426 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677"} err="failed to get container status \"5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677\": rpc error: code = NotFound desc = could not find container \"5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677\": container with ID starting with 5505d1a1c463d20460b75c709602447c79a065637436c02ef81cdcb2ca80f677 not found: ID does not exist" Aug 05 17:51:20.202776 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.202750 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l"] Aug 05 17:51:20.209014 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:20.208991 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-41ad7fbb-88c8bd465-gvc5l"] Aug 05 17:51:21.026659 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:21.026626 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5899e3bc-599b-4144-8c00-2c8bdac84706" path="/var/lib/kubelet/pods/5899e3bc-599b-4144-8c00-2c8bdac84706/volumes" Aug 05 17:51:21.026979 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:21.026964 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" path="/var/lib/kubelet/pods/7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6/volumes" Aug 05 17:51:22.268820 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:22.268784 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:51:24.496912 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.496866 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8"] Aug 05 17:51:24.497439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.497280 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" containerName="payload-processing" Aug 05 17:51:24.497439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.497293 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" containerName="payload-processing" Aug 05 17:51:24.497439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.497303 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5899e3bc-599b-4144-8c00-2c8bdac84706" containerName="revoke-keys" Aug 05 17:51:24.497439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.497309 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5899e3bc-599b-4144-8c00-2c8bdac84706" containerName="revoke-keys" Aug 05 17:51:24.497439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.497361 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5899e3bc-599b-4144-8c00-2c8bdac84706" containerName="revoke-keys" Aug 05 17:51:24.497439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.497373 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ed91077-4b42-42ae-a0f7-01ad5b0aa3b6" containerName="payload-processing" Aug 05 17:51:24.502962 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.502938 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.505196 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.505176 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1d0f63d4\"" Aug 05 17:51:24.505286 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.505201 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1d0f63d4-dockercfg-4t2cn\"" Aug 05 17:51:24.511028 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.511004 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8"] Aug 05 17:51:24.557182 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.557153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j5h4\" (UniqueName: \"kubernetes.io/projected/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-kube-api-access-9j5h4\") pod \"payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.557308 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.557188 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.658532 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.658502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j5h4\" (UniqueName: \"kubernetes.io/projected/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-kube-api-access-9j5h4\") pod \"payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.658743 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.658544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.659262 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.659242 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.665448 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.665424 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j5h4\" (UniqueName: \"kubernetes.io/projected/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-kube-api-access-9j5h4\") pod \"payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.814150 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.814113 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:24.946550 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:24.946526 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8"] Aug 05 17:51:24.948581 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:51:24.948554 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa937eb7_e169_4dd8_846d_5ad8bd66feb7.slice/crio-398b4b05f0c19cf27836b910c3b034aa83696b373cd1236c54c2c72cfd6b3040 WatchSource:0}: Error finding container 398b4b05f0c19cf27836b910c3b034aa83696b373cd1236c54c2c72cfd6b3040: Status 404 returned error can't find the container with id 398b4b05f0c19cf27836b910c3b034aa83696b373cd1236c54c2c72cfd6b3040 Aug 05 17:51:25.215766 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:25.215669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" event={"ID":"aa937eb7-e169-4dd8-846d-5ad8bd66feb7","Type":"ContainerStarted","Data":"bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8"} Aug 05 17:51:25.215944 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:25.215771 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:25.215944 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:25.215796 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" event={"ID":"aa937eb7-e169-4dd8-846d-5ad8bd66feb7","Type":"ContainerStarted","Data":"398b4b05f0c19cf27836b910c3b034aa83696b373cd1236c54c2c72cfd6b3040"} Aug 05 17:51:25.232344 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:25.232299 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" podStartSLOduration=1.232286311 podStartE2EDuration="1.232286311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:51:25.230528165 +0000 UTC m=+1922.631756979" watchObservedRunningTime="2026-08-05 17:51:25.232286311 +0000 UTC m=+1922.633515125" Aug 05 17:51:30.372955 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:30.372916 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:51:36.223673 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:36.223636 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:48.424918 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.424884 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747"] Aug 05 17:51:48.428643 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.428624 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:48.430570 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.430546 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 17:51:48.430682 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.430637 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 17:51:48.435576 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.435552 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747"] Aug 05 17:51:48.575114 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.575073 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7360bf07-fd9a-4537-b3b8-dc14f133cb74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747\" (UID: \"7360bf07-fd9a-4537-b3b8-dc14f133cb74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:48.676488 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.676406 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7360bf07-fd9a-4537-b3b8-dc14f133cb74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747\" (UID: \"7360bf07-fd9a-4537-b3b8-dc14f133cb74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:48.677045 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.677025 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7360bf07-fd9a-4537-b3b8-dc14f133cb74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747\" (UID: \"7360bf07-fd9a-4537-b3b8-dc14f133cb74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:48.740174 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.740134 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:48.875071 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:48.875013 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747"] Aug 05 17:51:48.880424 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:51:48.880398 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7360bf07_fd9a_4537_b3b8_dc14f133cb74.slice/crio-1701f3b95ec5b824f14207c476082be01e515cd7556d25c2b0c9bd49acf4b946 WatchSource:0}: Error finding container 1701f3b95ec5b824f14207c476082be01e515cd7556d25c2b0c9bd49acf4b946: Status 404 returned error can't find the container with id 1701f3b95ec5b824f14207c476082be01e515cd7556d25c2b0c9bd49acf4b946 Aug 05 17:51:49.347536 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:49.347502 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" event={"ID":"7360bf07-fd9a-4537-b3b8-dc14f133cb74","Type":"ContainerStarted","Data":"20d255e4292d224eb66ffeddbda51f56e26776e32e5db98bb80d45684f021d0d"} Aug 05 17:51:49.347724 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:49.347555 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" event={"ID":"7360bf07-fd9a-4537-b3b8-dc14f133cb74","Type":"ContainerStarted","Data":"1701f3b95ec5b824f14207c476082be01e515cd7556d25c2b0c9bd49acf4b946"} Aug 05 17:51:49.360851 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:49.360622 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" podStartSLOduration=1.360602704 podStartE2EDuration="1.360602704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:51:49.360338943 +0000 UTC m=+1946.761567757" watchObservedRunningTime="2026-08-05 17:51:49.360602704 +0000 UTC m=+1946.761831517" Aug 05 17:51:50.354708 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:50.354674 2575 generic.go:356] "Generic (PLEG): container finished" podID="7360bf07-fd9a-4537-b3b8-dc14f133cb74" containerID="20d255e4292d224eb66ffeddbda51f56e26776e32e5db98bb80d45684f021d0d" exitCode=0 Aug 05 17:51:50.355113 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:50.354714 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" event={"ID":"7360bf07-fd9a-4537-b3b8-dc14f133cb74","Type":"ContainerDied","Data":"20d255e4292d224eb66ffeddbda51f56e26776e32e5db98bb80d45684f021d0d"} Aug 05 17:51:51.489445 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:51.489421 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:51.603690 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:51.603663 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7360bf07-fd9a-4537-b3b8-dc14f133cb74-maas-api-service-ca\") pod \"7360bf07-fd9a-4537-b3b8-dc14f133cb74\" (UID: \"7360bf07-fd9a-4537-b3b8-dc14f133cb74\") " Aug 05 17:51:51.603996 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:51.603973 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7360bf07-fd9a-4537-b3b8-dc14f133cb74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7360bf07-fd9a-4537-b3b8-dc14f133cb74" (UID: "7360bf07-fd9a-4537-b3b8-dc14f133cb74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:51:51.705413 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:51.705343 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7360bf07-fd9a-4537-b3b8-dc14f133cb74-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:51:52.365374 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:52.365347 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" Aug 05 17:51:52.365573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:52.365396 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747" event={"ID":"7360bf07-fd9a-4537-b3b8-dc14f133cb74","Type":"ContainerDied","Data":"1701f3b95ec5b824f14207c476082be01e515cd7556d25c2b0c9bd49acf4b946"} Aug 05 17:51:52.365573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:52.365442 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1701f3b95ec5b824f14207c476082be01e515cd7556d25c2b0c9bd49acf4b946" Aug 05 17:51:54.313649 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.313617 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8"] Aug 05 17:51:54.314080 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.313852 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" podUID="aa937eb7-e169-4dd8-846d-5ad8bd66feb7" containerName="payload-processing" containerID="cri-o://bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8" gracePeriod=30 Aug 05 17:51:54.453079 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.452997 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747"] Aug 05 17:51:54.457287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.457259 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1d0f63d4-6v747"] Aug 05 17:51:54.572982 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.572922 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:54.735399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.735370 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j5h4\" (UniqueName: \"kubernetes.io/projected/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-kube-api-access-9j5h4\") pod \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " Aug 05 17:51:54.735399 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.735404 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-plugins-config-volume\") pod \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\" (UID: \"aa937eb7-e169-4dd8-846d-5ad8bd66feb7\") " Aug 05 17:51:54.735816 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.735788 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa937eb7-e169-4dd8-846d-5ad8bd66feb7" (UID: "aa937eb7-e169-4dd8-846d-5ad8bd66feb7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:51:54.737468 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.737443 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-kube-api-access-9j5h4" (OuterVolumeSpecName: "kube-api-access-9j5h4") pod "aa937eb7-e169-4dd8-846d-5ad8bd66feb7" (UID: "aa937eb7-e169-4dd8-846d-5ad8bd66feb7"). InnerVolumeSpecName "kube-api-access-9j5h4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:51:54.836767 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.836678 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j5h4\" (UniqueName: \"kubernetes.io/projected/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-kube-api-access-9j5h4\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:51:54.836767 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:54.836717 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa937eb7-e169-4dd8-846d-5ad8bd66feb7-plugins-config-volume\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:51:55.032837 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.032798 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7360bf07-fd9a-4537-b3b8-dc14f133cb74" path="/var/lib/kubelet/pods/7360bf07-fd9a-4537-b3b8-dc14f133cb74/volumes" Aug 05 17:51:55.381665 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.381631 2575 generic.go:356] "Generic (PLEG): container finished" podID="aa937eb7-e169-4dd8-846d-5ad8bd66feb7" containerID="bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8" exitCode=0 Aug 05 17:51:55.382169 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.381688 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" Aug 05 17:51:55.382169 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.381717 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" event={"ID":"aa937eb7-e169-4dd8-846d-5ad8bd66feb7","Type":"ContainerDied","Data":"bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8"} Aug 05 17:51:55.382169 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.381778 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8" event={"ID":"aa937eb7-e169-4dd8-846d-5ad8bd66feb7","Type":"ContainerDied","Data":"398b4b05f0c19cf27836b910c3b034aa83696b373cd1236c54c2c72cfd6b3040"} Aug 05 17:51:55.382169 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.381813 2575 scope.go:117] "RemoveContainer" containerID="bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8" Aug 05 17:51:55.390730 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.390714 2575 scope.go:117] "RemoveContainer" containerID="bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8" Aug 05 17:51:55.390979 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:51:55.390959 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8\": container with ID starting with bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8 not found: ID does not exist" containerID="bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8" Aug 05 17:51:55.391029 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.390986 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8"} err="failed to get container status \"bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8\": rpc error: code = NotFound desc = could not find container \"bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8\": container with ID starting with bc4df59f8385f0a87b6cb44ccfb89d61b73e3986f9d4b2f1344b47c5a76006e8 not found: ID does not exist" Aug 05 17:51:55.399588 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.399568 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8"] Aug 05 17:51:55.404775 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:55.404754 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d0f63d4-7fbbd9dbbd-6pvb8"] Aug 05 17:51:56.343825 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.343796 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6"] Aug 05 17:51:56.344194 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.344181 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa937eb7-e169-4dd8-846d-5ad8bd66feb7" containerName="payload-processing" Aug 05 17:51:56.344245 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.344196 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa937eb7-e169-4dd8-846d-5ad8bd66feb7" containerName="payload-processing" Aug 05 17:51:56.344245 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.344214 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7360bf07-fd9a-4537-b3b8-dc14f133cb74" containerName="revoke-keys" Aug 05 17:51:56.344245 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.344220 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7360bf07-fd9a-4537-b3b8-dc14f133cb74" containerName="revoke-keys" Aug 05 17:51:56.344355 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.344281 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa937eb7-e169-4dd8-846d-5ad8bd66feb7" containerName="payload-processing" Aug 05 17:51:56.344355 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.344290 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7360bf07-fd9a-4537-b3b8-dc14f133cb74" containerName="revoke-keys" Aug 05 17:51:56.347192 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.347176 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.349159 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.349136 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 05 17:51:56.349282 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.349227 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-12e4a979-openshift-default-dockercfg-ss8hw\"" Aug 05 17:51:56.359228 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.359208 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6"] Aug 05 17:51:56.450490 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450455 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-data\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450490 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450486 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istiod-ca-cert\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450508 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbrd\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-kube-api-access-lpbrd\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450601 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-envoy\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450676 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-certs\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450708 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-token\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450760 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-credential-socket\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-socket\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.450974 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.450891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-podinfo\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551403 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-credential-socket\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551611 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-socket\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551611 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551481 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-podinfo\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551611 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-data\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551611 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551538 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istiod-ca-cert\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551611 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551568 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbrd\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-kube-api-access-lpbrd\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551893 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551638 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-envoy\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551893 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551803 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-credential-socket\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.551893 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551846 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-socket\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.552072 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551898 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-certs\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.552072 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551923 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-data\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.552072 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.551971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-token\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.552250 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.552204 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-certs\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.552308 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.552282 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istiod-ca-cert\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.553928 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.553906 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-envoy\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.554276 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.554255 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-podinfo\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.559580 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.559551 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbrd\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-kube-api-access-lpbrd\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.559756 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.559736 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-token\") pod \"e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.658003 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.657917 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:51:56.797181 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:56.797155 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6"] Aug 05 17:51:56.798655 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:51:56.798631 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a291e33_11fe_4d1b_bfb2_d2bc1bf19e06.slice/crio-67351080b7e4cc8610b24c1b3868e4f512ba78071ed27320dec92eb325049ca3 WatchSource:0}: Error finding container 67351080b7e4cc8610b24c1b3868e4f512ba78071ed27320dec92eb325049ca3: Status 404 returned error can't find the container with id 67351080b7e4cc8610b24c1b3868e4f512ba78071ed27320dec92eb325049ca3 Aug 05 17:51:57.029884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:57.029801 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa937eb7-e169-4dd8-846d-5ad8bd66feb7" path="/var/lib/kubelet/pods/aa937eb7-e169-4dd8-846d-5ad8bd66feb7/volumes" Aug 05 17:51:57.068434 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:57.068407 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:51:57.395439 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:57.395406 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" event={"ID":"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06","Type":"ContainerStarted","Data":"67351080b7e4cc8610b24c1b3868e4f512ba78071ed27320dec92eb325049ca3"} Aug 05 17:51:59.602587 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:59.602546 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 17:51:59.602881 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:59.602619 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 17:51:59.602881 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:51:59.602649 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 17:52:00.428609 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:00.428572 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" event={"ID":"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06","Type":"ContainerStarted","Data":"e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314"} Aug 05 17:52:00.448824 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:00.448770 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podStartSLOduration=1.647159728 podStartE2EDuration="4.448755574s" totalImagesPullingTime="2.801595846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:51:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:52:00.446000806 +0000 UTC m=+1957.847229621" watchObservedRunningTime="2026-08-05 17:52:00.448755574 +0000 UTC m=+1957.849984387" Aug 05 17:52:00.658388 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:00.658348 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:00.663613 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:00.663584 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:01.434213 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:01.434171 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:01.435173 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:01.435148 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:06.070542 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:06.070507 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:52:15.512386 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:15.512351 2575 generic.go:356] "Generic (PLEG): container finished" podID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerID="e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314" exitCode=0 Aug 05 17:52:15.512807 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:15.512422 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" event={"ID":"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06","Type":"ContainerDied","Data":"e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314"} Aug 05 17:52:15.512807 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:15.512801 2575 scope.go:117] "RemoveContainer" containerID="e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314" Aug 05 17:52:15.514707 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:15.514657 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 17:52:15.514797 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:15.514745 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 17:52:15.514797 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:15.514775 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 17:52:16.434997 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:16.434947 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:16.520379 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:16.520341 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" event={"ID":"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06","Type":"ContainerStarted","Data":"8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984"} Aug 05 17:52:16.659507 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:16.659470 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:16.660224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:16.659879 2575 patch_prober.go:28] interesting pod/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Aug 05 17:52:16.660224 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:16.659940 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Aug 05 17:52:17.659315 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:17.659269 2575 patch_prober.go:28] interesting pod/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Aug 05 17:52:17.659880 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:17.659347 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Aug 05 17:52:18.658514 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:18.658474 2575 patch_prober.go:28] interesting pod/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Aug 05 17:52:18.658861 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:18.658824 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Aug 05 17:52:19.658976 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:19.658934 2575 patch_prober.go:28] interesting pod/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Aug 05 17:52:19.659739 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:19.659702 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Aug 05 17:52:20.659758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:20.659316 2575 patch_prober.go:28] interesting pod/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Aug 05 17:52:20.659758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:20.659384 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Aug 05 17:52:21.662879 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:21.662843 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:21.663281 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:21.663121 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:21.663885 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:21.663869 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:43.722930 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:43.722891 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6"] Aug 05 17:52:43.723804 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:43.723749 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" containerID="cri-o://8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984" gracePeriod=30 Aug 05 17:52:46.275095 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:46.275047 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:52:48.974217 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:48.974194 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:49.020212 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020138 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-podinfo\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020212 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020179 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-socket\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020212 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020207 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpbrd\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-kube-api-access-lpbrd\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020462 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020235 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-data\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020462 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020253 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-token\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020462 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020275 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-envoy\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020462 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020305 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-certs\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020462 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020339 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-credential-socket\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020462 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020427 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istiod-ca-cert\") pod \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\" (UID: \"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06\") " Aug 05 17:52:49.020781 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020520 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:52:49.020781 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.020690 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-data\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.021713 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.021422 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:52:49.021713 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.021495 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:52:49.021713 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.021542 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:52:49.022043 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.022008 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:52:49.023670 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.023331 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-kube-api-access-lpbrd" (OuterVolumeSpecName: "kube-api-access-lpbrd") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "kube-api-access-lpbrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:52:49.025654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.024146 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 17:52:49.025654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.024802 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 05 17:52:49.032274 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.032226 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" (UID: "1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 17:52:49.121573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121544 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-credential-socket\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121571 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istiod-ca-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121573 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121580 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-podinfo\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121589 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-socket\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121597 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpbrd\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-kube-api-access-lpbrd\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121606 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-token\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121614 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-istio-envoy\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.121787 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.121622 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06-workload-certs\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:52:49.712835 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.712798 2575 generic.go:356] "Generic (PLEG): container finished" podID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerID="8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984" exitCode=0 Aug 05 17:52:49.712985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.712858 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" Aug 05 17:52:49.712985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.712864 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" event={"ID":"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06","Type":"ContainerDied","Data":"8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984"} Aug 05 17:52:49.712985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.712909 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6" event={"ID":"1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06","Type":"ContainerDied","Data":"67351080b7e4cc8610b24c1b3868e4f512ba78071ed27320dec92eb325049ca3"} Aug 05 17:52:49.712985 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.712931 2575 scope.go:117] "RemoveContainer" containerID="8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984" Aug 05 17:52:49.725935 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.722910 2575 scope.go:117] "RemoveContainer" containerID="e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314" Aug 05 17:52:49.731828 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.731801 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6"] Aug 05 17:52:49.732672 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.732656 2575 scope.go:117] "RemoveContainer" containerID="8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984" Aug 05 17:52:49.732937 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:52:49.732913 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984\": container with ID starting with 8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984 not found: ID does not exist" containerID="8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984" Aug 05 17:52:49.733064 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.732943 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984"} err="failed to get container status \"8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984\": rpc error: code = NotFound desc = could not find container \"8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984\": container with ID starting with 8cc6a57bd3bc1f125387de403586cf8d939662f4104d3d1af108198fbd4a0984 not found: ID does not exist" Aug 05 17:52:49.733064 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.732961 2575 scope.go:117] "RemoveContainer" containerID="e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314" Aug 05 17:52:49.733522 ip-10-0-143-58 kubenswrapper[2575]: E0805 17:52:49.733211 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314\": container with ID starting with e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314 not found: ID does not exist" containerID="e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314" Aug 05 17:52:49.733522 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.733287 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314"} err="failed to get container status \"e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314\": rpc error: code = NotFound desc = could not find container \"e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314\": container with ID starting with e20fc9dccfa85ee1b7d09f9957bc1fc0b09e73490bec573d1458efc4191e8314 not found: ID does not exist" Aug 05 17:52:49.736074 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:49.736043 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-12e4a979-openshift-default-56d7ff45b4-69mh6"] Aug 05 17:52:51.025994 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:51.025947 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" path="/var/lib/kubelet/pods/1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06/volumes" Aug 05 17:52:57.763418 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:52:57.763383 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:54:23.198497 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:23.198464 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:54:23.215022 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:23.214989 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:54:23.224934 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:23.224906 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:54:23.231887 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:23.231860 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:54:26.777608 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:26.777576 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:54:32.069732 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:32.069694 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:54:39.574945 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:39.574907 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:54:49.268866 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:54:49.268826 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:55:17.427383 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427306 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv"] Aug 05 17:55:17.427829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427730 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" Aug 05 17:55:17.427829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427743 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" Aug 05 17:55:17.427829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427753 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" Aug 05 17:55:17.427829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427759 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" Aug 05 17:55:17.427829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427830 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" Aug 05 17:55:17.427991 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.427841 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a291e33-11fe-4d1b-bfb2-d2bc1bf19e06" containerName="istio-proxy" Aug 05 17:55:17.430975 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.430960 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:17.433420 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.433391 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 17:55:17.433544 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.433462 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 17:55:17.439257 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.439232 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv"] Aug 05 17:55:17.464123 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.464094 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv\" (UID: \"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:17.565324 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.565291 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv\" (UID: \"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:17.565921 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.565903 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv\" (UID: \"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:17.742207 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.742114 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:17.873219 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:17.873192 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv"] Aug 05 17:55:17.875214 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:55:17.875192 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fadd85_a36d_4cbf_bd1a_e9a8e139b76c.slice/crio-0113ce6d8e9f977d8ab3b81c481206d203f456b1e7fa1639570e12f160c1f616 WatchSource:0}: Error finding container 0113ce6d8e9f977d8ab3b81c481206d203f456b1e7fa1639570e12f160c1f616: Status 404 returned error can't find the container with id 0113ce6d8e9f977d8ab3b81c481206d203f456b1e7fa1639570e12f160c1f616 Aug 05 17:55:18.542179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:18.542086 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" event={"ID":"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c","Type":"ContainerStarted","Data":"f7f66307fffbb88e6544d36cb4ff976a63be11a45bff039d517a6bd840d0e00b"} Aug 05 17:55:18.542179 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:18.542143 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" event={"ID":"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c","Type":"ContainerStarted","Data":"0113ce6d8e9f977d8ab3b81c481206d203f456b1e7fa1639570e12f160c1f616"} Aug 05 17:55:18.556975 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:18.556929 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" podStartSLOduration=1.556913973 podStartE2EDuration="1.556913973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:55:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:55:18.554722679 +0000 UTC m=+2155.955951495" watchObservedRunningTime="2026-08-05 17:55:18.556913973 +0000 UTC m=+2155.958142832" Aug 05 17:55:19.549654 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:19.549620 2575 generic.go:356] "Generic (PLEG): container finished" podID="85fadd85-a36d-4cbf-bd1a-e9a8e139b76c" containerID="f7f66307fffbb88e6544d36cb4ff976a63be11a45bff039d517a6bd840d0e00b" exitCode=0 Aug 05 17:55:19.550040 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:19.549664 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" event={"ID":"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c","Type":"ContainerDied","Data":"f7f66307fffbb88e6544d36cb4ff976a63be11a45bff039d517a6bd840d0e00b"} Aug 05 17:55:20.681566 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:20.681536 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:20.791963 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:20.791925 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c-maas-api-service-ca\") pod \"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c\" (UID: \"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c\") " Aug 05 17:55:20.792372 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:20.792339 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85fadd85-a36d-4cbf-bd1a-e9a8e139b76c" (UID: "85fadd85-a36d-4cbf-bd1a-e9a8e139b76c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:55:20.892951 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:20.892866 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:55:21.561586 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:21.561556 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" Aug 05 17:55:21.561758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:21.561551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv" event={"ID":"85fadd85-a36d-4cbf-bd1a-e9a8e139b76c","Type":"ContainerDied","Data":"0113ce6d8e9f977d8ab3b81c481206d203f456b1e7fa1639570e12f160c1f616"} Aug 05 17:55:21.561758 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:21.561685 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0113ce6d8e9f977d8ab3b81c481206d203f456b1e7fa1639570e12f160c1f616" Aug 05 17:55:28.098356 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:28.098324 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv"] Aug 05 17:55:28.100359 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:28.100336 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f48a67f-7mtdv"] Aug 05 17:55:29.030204 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:29.030158 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85fadd85-a36d-4cbf-bd1a-e9a8e139b76c" path="/var/lib/kubelet/pods/85fadd85-a36d-4cbf-bd1a-e9a8e139b76c/volumes" Aug 05 17:55:30.068499 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:55:30.068464 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:56:05.974497 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:56:05.974453 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:57:23.257177 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:57:23.257140 2575 scope.go:117] "RemoveContainer" containerID="dc0c26ba5aca7c82221d9f2359d4f5b41a985fe15584fb572a43599e19ebc1a8" Aug 05 17:57:23.265914 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:57:23.265889 2575 scope.go:117] "RemoveContainer" containerID="e505afd504c9b99a632a24f3a3a3cd9dbded3419bc3ddcaf77f30d1eb0b22e23" Aug 05 17:57:53.065119 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:57:53.065081 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:58:01.068895 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:58:01.068857 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:58:23.289884 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:58:23.289791 2575 scope.go:117] "RemoveContainer" containerID="20d255e4292d224eb66ffeddbda51f56e26776e32e5db98bb80d45684f021d0d" Aug 05 17:58:40.469076 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:58:40.469018 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:58:53.559637 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:58:53.559598 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:59:02.886523 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.886488 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt"] Aug 05 17:59:02.886915 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.886883 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85fadd85-a36d-4cbf-bd1a-e9a8e139b76c" containerName="revoke-keys" Aug 05 17:59:02.886915 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.886895 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="85fadd85-a36d-4cbf-bd1a-e9a8e139b76c" containerName="revoke-keys" Aug 05 17:59:02.886989 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.886973 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="85fadd85-a36d-4cbf-bd1a-e9a8e139b76c" containerName="revoke-keys" Aug 05 17:59:02.889918 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.889902 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:02.892148 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.892122 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 17:59:02.892287 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.892131 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 17:59:02.898447 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.898423 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt"] Aug 05 17:59:02.967409 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:02.967380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76622a8-c829-465d-b0cc-d83dee9bab72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt\" (UID: \"b76622a8-c829-465d-b0cc-d83dee9bab72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:03.068124 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.068088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76622a8-c829-465d-b0cc-d83dee9bab72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt\" (UID: \"b76622a8-c829-465d-b0cc-d83dee9bab72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:03.068691 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.068673 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76622a8-c829-465d-b0cc-d83dee9bab72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt\" (UID: \"b76622a8-c829-465d-b0cc-d83dee9bab72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:03.202777 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.202702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:03.329198 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.329172 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt"] Aug 05 17:59:03.331758 ip-10-0-143-58 kubenswrapper[2575]: W0805 17:59:03.331723 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76622a8_c829_465d_b0cc_d83dee9bab72.slice/crio-e8b32350c7ec48de867a65a7b434e84212939b5b04dd5eb6c2b7684277b09d72 WatchSource:0}: Error finding container e8b32350c7ec48de867a65a7b434e84212939b5b04dd5eb6c2b7684277b09d72: Status 404 returned error can't find the container with id e8b32350c7ec48de867a65a7b434e84212939b5b04dd5eb6c2b7684277b09d72 Aug 05 17:59:03.793326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.793231 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" event={"ID":"b76622a8-c829-465d-b0cc-d83dee9bab72","Type":"ContainerStarted","Data":"578f96be25e6c3d757619323784e9ef4296653ce6b1de650e08df1a00b319991"} Aug 05 17:59:03.793326 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.793314 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" event={"ID":"b76622a8-c829-465d-b0cc-d83dee9bab72","Type":"ContainerStarted","Data":"e8b32350c7ec48de867a65a7b434e84212939b5b04dd5eb6c2b7684277b09d72"} Aug 05 17:59:03.808726 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:03.808677 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" podStartSLOduration=1.808663938 podStartE2EDuration="1.808663938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:59:03.807374787 +0000 UTC m=+2381.208603603" watchObservedRunningTime="2026-08-05 17:59:03.808663938 +0000 UTC m=+2381.209892794" Aug 05 17:59:04.800829 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:04.800787 2575 generic.go:356] "Generic (PLEG): container finished" podID="b76622a8-c829-465d-b0cc-d83dee9bab72" containerID="578f96be25e6c3d757619323784e9ef4296653ce6b1de650e08df1a00b319991" exitCode=0 Aug 05 17:59:04.801238 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:04.800854 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" event={"ID":"b76622a8-c829-465d-b0cc-d83dee9bab72","Type":"ContainerDied","Data":"578f96be25e6c3d757619323784e9ef4296653ce6b1de650e08df1a00b319991"} Aug 05 17:59:05.933527 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:05.933498 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:06.094799 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:06.094706 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76622a8-c829-465d-b0cc-d83dee9bab72-maas-api-service-ca\") pod \"b76622a8-c829-465d-b0cc-d83dee9bab72\" (UID: \"b76622a8-c829-465d-b0cc-d83dee9bab72\") " Aug 05 17:59:06.095113 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:06.095088 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b76622a8-c829-465d-b0cc-d83dee9bab72-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b76622a8-c829-465d-b0cc-d83dee9bab72" (UID: "b76622a8-c829-465d-b0cc-d83dee9bab72"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 17:59:06.195669 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:06.195636 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76622a8-c829-465d-b0cc-d83dee9bab72-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 17:59:06.814316 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:06.814283 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" event={"ID":"b76622a8-c829-465d-b0cc-d83dee9bab72","Type":"ContainerDied","Data":"e8b32350c7ec48de867a65a7b434e84212939b5b04dd5eb6c2b7684277b09d72"} Aug 05 17:59:06.814316 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:06.814308 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt" Aug 05 17:59:06.814520 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:06.814331 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8b32350c7ec48de867a65a7b434e84212939b5b04dd5eb6c2b7684277b09d72" Aug 05 17:59:16.169178 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:16.169141 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 17:59:18.755615 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:18.755575 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt"] Aug 05 17:59:18.757640 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:18.757616 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b60f38ad-hrtzt"] Aug 05 17:59:19.026734 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:19.026655 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76622a8-c829-465d-b0cc-d83dee9bab72" path="/var/lib/kubelet/pods/b76622a8-c829-465d-b0cc-d83dee9bab72/volumes" Aug 05 17:59:23.249359 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:23.249327 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:59:23.255422 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:23.255403 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:59:23.267855 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:23.267837 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 17:59:23.273248 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:23.273230 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 17:59:23.767562 ip-10-0-143-58 kubenswrapper[2575]: I0805 17:59:23.767529 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:00:00.135409 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.135368 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx"] Aug 05 18:00:00.135870 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.135759 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76622a8-c829-465d-b0cc-d83dee9bab72" containerName="revoke-keys" Aug 05 18:00:00.135870 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.135771 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76622a8-c829-465d-b0cc-d83dee9bab72" containerName="revoke-keys" Aug 05 18:00:00.135870 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.135845 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76622a8-c829-465d-b0cc-d83dee9bab72" containerName="revoke-keys" Aug 05 18:00:00.138814 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.138796 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" Aug 05 18:00:00.140943 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.140925 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:00:00.145217 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.145192 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx"] Aug 05 18:00:00.150568 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.150550 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" Aug 05 18:00:00.491013 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:00.490983 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx"] Aug 05 18:00:00.492945 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:00:00.492913 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598c18c4_6f79_46af_8006_0e98777be0b9.slice/crio-59c76404ceeb68c99726359a520b1509026903875a2d377d40dbcf5321d49da2 WatchSource:0}: Error finding container 59c76404ceeb68c99726359a520b1509026903875a2d377d40dbcf5321d49da2: Status 404 returned error can't find the container with id 59c76404ceeb68c99726359a520b1509026903875a2d377d40dbcf5321d49da2 Aug 05 18:00:01.114026 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:01.113985 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" event={"ID":"598c18c4-6f79-46af-8006-0e98777be0b9","Type":"ContainerStarted","Data":"a24a42cec2c04e1d954cbebb991fa3e71fb2cac4a20f5902633ede00f8106652"} Aug 05 18:00:01.114026 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:01.114036 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" event={"ID":"598c18c4-6f79-46af-8006-0e98777be0b9","Type":"ContainerStarted","Data":"59c76404ceeb68c99726359a520b1509026903875a2d377d40dbcf5321d49da2"} Aug 05 18:00:01.131830 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:01.131772 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" podStartSLOduration=1.131752677 podStartE2EDuration="1.131752677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:00:01.127876179 +0000 UTC m=+2438.529104993" watchObservedRunningTime="2026-08-05 18:00:01.131752677 +0000 UTC m=+2438.532981494" Aug 05 18:00:02.120508 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:02.120476 2575 generic.go:356] "Generic (PLEG): container finished" podID="598c18c4-6f79-46af-8006-0e98777be0b9" containerID="a24a42cec2c04e1d954cbebb991fa3e71fb2cac4a20f5902633ede00f8106652" exitCode=0 Aug 05 18:00:02.120916 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:02.120567 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" event={"ID":"598c18c4-6f79-46af-8006-0e98777be0b9","Type":"ContainerDied","Data":"a24a42cec2c04e1d954cbebb991fa3e71fb2cac4a20f5902633ede00f8106652"} Aug 05 18:00:03.255658 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:03.255629 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" Aug 05 18:00:04.131545 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:04.131516 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" Aug 05 18:00:04.131744 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:04.131507 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765880-zshdx" event={"ID":"598c18c4-6f79-46af-8006-0e98777be0b9","Type":"ContainerDied","Data":"59c76404ceeb68c99726359a520b1509026903875a2d377d40dbcf5321d49da2"} Aug 05 18:00:04.131744 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:04.131637 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59c76404ceeb68c99726359a520b1509026903875a2d377d40dbcf5321d49da2" Aug 05 18:00:11.677181 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:11.677142 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:00:24.071089 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:24.071043 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:00:31.765606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:00:31.765568 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:01:23.329573 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:23.329496 2575 scope.go:117] "RemoveContainer" containerID="f7f66307fffbb88e6544d36cb4ff976a63be11a45bff039d517a6bd840d0e00b" Aug 05 18:01:23.867030 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:23.866995 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:01:44.535210 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.535170 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf"] Aug 05 18:01:44.535857 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.535840 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598c18c4-6f79-46af-8006-0e98777be0b9" containerName="cleanup" Aug 05 18:01:44.535912 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.535860 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="598c18c4-6f79-46af-8006-0e98777be0b9" containerName="cleanup" Aug 05 18:01:44.535964 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.535954 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="598c18c4-6f79-46af-8006-0e98777be0b9" containerName="cleanup" Aug 05 18:01:44.539000 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.538983 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.541390 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.541374 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-76d454-openshift-default-dockercfg-gwmwt\"" Aug 05 18:01:44.541390 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.541384 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 05 18:01:44.549747 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.549725 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf"] Aug 05 18:01:44.607728 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5llgj\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-kube-api-access-5llgj\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.607728 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-envoy\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.607964 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607758 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-socket\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.607964 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607811 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-data\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.607964 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607831 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-token\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.607964 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-credential-socket\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.607964 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607901 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-podinfo\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.608151 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.607954 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d93bcf-992d-40bd-95b8-c0e38724724f-istiod-ca-cert\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.608151 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.608004 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-certs\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709350 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5llgj\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-kube-api-access-5llgj\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709350 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-envoy\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-socket\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-data\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709421 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-token\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-credential-socket\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-podinfo\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709497 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d93bcf-992d-40bd-95b8-c0e38724724f-istiod-ca-cert\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709546 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-certs\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709970 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709822 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-socket\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709970 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709874 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-data\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709970 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709911 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-certs\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.709970 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.709945 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-credential-socket\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.710337 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.710318 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d93bcf-992d-40bd-95b8-c0e38724724f-istiod-ca-cert\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.711665 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.711638 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-envoy\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.712078 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.712027 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-podinfo\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.717375 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.717349 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-token\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.717694 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.717669 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5llgj\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-kube-api-access-5llgj\") pod \"e2e-inf-b-76d454-openshift-default-88b588864-kmfdf\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.850724 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.850683 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:44.993288 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.993254 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf"] Aug 05 18:01:44.994555 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:01:44.994519 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d93bcf_992d_40bd_95b8_c0e38724724f.slice/crio-4be51d04a4b6a4d326424326f7ca9a77e5a41726bc4c6791aaae9303f87b74a3 WatchSource:0}: Error finding container 4be51d04a4b6a4d326424326f7ca9a77e5a41726bc4c6791aaae9303f87b74a3: Status 404 returned error can't find the container with id 4be51d04a4b6a4d326424326f7ca9a77e5a41726bc4c6791aaae9303f87b74a3 Aug 05 18:01:44.996734 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.996699 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:01:44.996830 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.996781 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:01:44.996903 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:44.996829 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:01:45.702627 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:45.702591 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" event={"ID":"44d93bcf-992d-40bd-95b8-c0e38724724f","Type":"ContainerStarted","Data":"69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4"} Aug 05 18:01:45.703026 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:45.702646 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" event={"ID":"44d93bcf-992d-40bd-95b8-c0e38724724f","Type":"ContainerStarted","Data":"4be51d04a4b6a4d326424326f7ca9a77e5a41726bc4c6791aaae9303f87b74a3"} Aug 05 18:01:45.722516 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:45.722470 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podStartSLOduration=1.722454908 podStartE2EDuration="1.722454908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:45.7207487 +0000 UTC m=+2543.121977516" watchObservedRunningTime="2026-08-05 18:01:45.722454908 +0000 UTC m=+2543.123683723" Aug 05 18:01:45.850920 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:45.850878 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:46.856447 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:46.856415 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:47.714721 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:47.714687 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:47.715718 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:47.715696 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:01:49.409190 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.409156 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx"] Aug 05 18:01:49.412886 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.412863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.415179 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.415157 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-76d454-dockercfg-w466r\"" Aug 05 18:01:49.415286 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.415162 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-76d454\"" Aug 05 18:01:49.429159 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.429132 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx"] Aug 05 18:01:49.455468 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.455435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j9d2\" (UniqueName: \"kubernetes.io/projected/ea41d039-d0ee-4b61-aa0c-0d4a74127026-kube-api-access-6j9d2\") pod \"payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.455658 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.455508 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea41d039-d0ee-4b61-aa0c-0d4a74127026-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.556985 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.556944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea41d039-d0ee-4b61-aa0c-0d4a74127026-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.557228 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.557026 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j9d2\" (UniqueName: \"kubernetes.io/projected/ea41d039-d0ee-4b61-aa0c-0d4a74127026-kube-api-access-6j9d2\") pod \"payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.557782 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.557756 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea41d039-d0ee-4b61-aa0c-0d4a74127026-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.566028 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.566005 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j9d2\" (UniqueName: \"kubernetes.io/projected/ea41d039-d0ee-4b61-aa0c-0d4a74127026-kube-api-access-6j9d2\") pod \"payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.728258 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.728155 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:49.870945 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:49.870912 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx"] Aug 05 18:01:49.872896 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:01:49.872866 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea41d039_d0ee_4b61_aa0c_0d4a74127026.slice/crio-5a1ea72f16f50086078570adfb550df00334d820f0c41ea9cadfa9b52f89ffed WatchSource:0}: Error finding container 5a1ea72f16f50086078570adfb550df00334d820f0c41ea9cadfa9b52f89ffed: Status 404 returned error can't find the container with id 5a1ea72f16f50086078570adfb550df00334d820f0c41ea9cadfa9b52f89ffed Aug 05 18:01:50.732647 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:50.732611 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" event={"ID":"ea41d039-d0ee-4b61-aa0c-0d4a74127026","Type":"ContainerStarted","Data":"9d944dd00db5046f1316891d53cb9dff569896093b4d7119ea574e492e62eb44"} Aug 05 18:01:50.733030 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:50.732667 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" event={"ID":"ea41d039-d0ee-4b61-aa0c-0d4a74127026","Type":"ContainerStarted","Data":"5a1ea72f16f50086078570adfb550df00334d820f0c41ea9cadfa9b52f89ffed"} Aug 05 18:01:50.733030 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:50.732697 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:01:50.750528 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:50.750475 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" podStartSLOduration=1.750455543 podStartE2EDuration="1.750455543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:50.746492419 +0000 UTC m=+2548.147721238" watchObservedRunningTime="2026-08-05 18:01:50.750455543 +0000 UTC m=+2548.151684360" Aug 05 18:01:51.473218 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:01:51.473179 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:02:01.741716 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.741626 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:02:01.813771 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.813738 2575 generic.go:356] "Generic (PLEG): container finished" podID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerID="69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4" exitCode=0 Aug 05 18:02:01.813954 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.813814 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" event={"ID":"44d93bcf-992d-40bd-95b8-c0e38724724f","Type":"ContainerDied","Data":"69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4"} Aug 05 18:02:01.814267 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.814251 2575 scope.go:117] "RemoveContainer" containerID="69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4" Aug 05 18:02:01.815311 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.815280 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:02:01.815378 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.815356 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:02:01.815420 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:01.815393 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:02:02.714757 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:02.714714 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:02:02.821752 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:02.821714 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" event={"ID":"44d93bcf-992d-40bd-95b8-c0e38724724f","Type":"ContainerStarted","Data":"341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff"} Aug 05 18:02:02.861294 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:02.861256 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:02:02.862260 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:02.862223 2575 patch_prober.go:28] interesting pod/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 05 18:02:02.862382 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:02.862297 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 05 18:02:03.851746 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:03.851708 2575 patch_prober.go:28] interesting pod/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 05 18:02:03.852382 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:03.852345 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 05 18:02:04.851575 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:04.851524 2575 patch_prober.go:28] interesting pod/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 05 18:02:04.851798 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:04.851616 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 05 18:02:05.851661 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:05.851613 2575 patch_prober.go:28] interesting pod/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 05 18:02:05.851894 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:05.851701 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 05 18:02:06.852224 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:06.852187 2575 patch_prober.go:28] interesting pod/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 05 18:02:06.852805 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:06.852772 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 05 18:02:07.877536 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:07.877498 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:02:07.878084 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:07.877899 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:02:07.878756 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:07.878731 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:02:20.972382 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:20.972340 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:02:53.661352 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:02:53.661308 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:03:03.169665 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:03.169619 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:03:33.376116 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.376073 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs"] Aug 05 18:03:33.379719 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.379694 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:33.381771 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.381751 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:03:33.381867 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.381756 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 18:03:33.387115 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.387091 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs"] Aug 05 18:03:33.525222 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.525183 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc34b1b-977b-4209-9a27-8ae950b5e482-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs\" (UID: \"9fc34b1b-977b-4209-9a27-8ae950b5e482\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:33.626851 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.626765 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc34b1b-977b-4209-9a27-8ae950b5e482-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs\" (UID: \"9fc34b1b-977b-4209-9a27-8ae950b5e482\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:33.627445 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.627423 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc34b1b-977b-4209-9a27-8ae950b5e482-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs\" (UID: \"9fc34b1b-977b-4209-9a27-8ae950b5e482\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:33.691562 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.691533 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:33.827706 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:03:33.827673 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc34b1b_977b_4209_9a27_8ae950b5e482.slice/crio-7e0e8c68e6b4ff5929cb9f7d338dd3e83b6c6e76245dc7eae7afd4af8d8fabcb WatchSource:0}: Error finding container 7e0e8c68e6b4ff5929cb9f7d338dd3e83b6c6e76245dc7eae7afd4af8d8fabcb: Status 404 returned error can't find the container with id 7e0e8c68e6b4ff5929cb9f7d338dd3e83b6c6e76245dc7eae7afd4af8d8fabcb Aug 05 18:03:33.827933 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:33.827910 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs"] Aug 05 18:03:34.349393 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:34.349298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" event={"ID":"9fc34b1b-977b-4209-9a27-8ae950b5e482","Type":"ContainerStarted","Data":"1cae8be741ee6236f3041975843d0187ce32b94bdb782a84bcf6a42b7b45f9af"} Aug 05 18:03:34.349393 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:34.349360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" event={"ID":"9fc34b1b-977b-4209-9a27-8ae950b5e482","Type":"ContainerStarted","Data":"7e0e8c68e6b4ff5929cb9f7d338dd3e83b6c6e76245dc7eae7afd4af8d8fabcb"} Aug 05 18:03:34.366441 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:34.366391 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" podStartSLOduration=1.366375301 podStartE2EDuration="1.366375301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:03:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:03:34.364505594 +0000 UTC m=+2651.765734418" watchObservedRunningTime="2026-08-05 18:03:34.366375301 +0000 UTC m=+2651.767604115" Aug 05 18:03:35.356402 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:35.356358 2575 generic.go:356] "Generic (PLEG): container finished" podID="9fc34b1b-977b-4209-9a27-8ae950b5e482" containerID="1cae8be741ee6236f3041975843d0187ce32b94bdb782a84bcf6a42b7b45f9af" exitCode=0 Aug 05 18:03:35.356802 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:35.356451 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" event={"ID":"9fc34b1b-977b-4209-9a27-8ae950b5e482","Type":"ContainerDied","Data":"1cae8be741ee6236f3041975843d0187ce32b94bdb782a84bcf6a42b7b45f9af"} Aug 05 18:03:35.568225 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:35.568191 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:03:36.494640 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:36.494617 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:36.555329 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:36.555303 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc34b1b-977b-4209-9a27-8ae950b5e482-maas-api-service-ca\") pod \"9fc34b1b-977b-4209-9a27-8ae950b5e482\" (UID: \"9fc34b1b-977b-4209-9a27-8ae950b5e482\") " Aug 05 18:03:36.555618 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:36.555597 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fc34b1b-977b-4209-9a27-8ae950b5e482-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9fc34b1b-977b-4209-9a27-8ae950b5e482" (UID: "9fc34b1b-977b-4209-9a27-8ae950b5e482"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:03:36.656402 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:36.656322 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fc34b1b-977b-4209-9a27-8ae950b5e482-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:03:37.369458 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:37.369428 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" Aug 05 18:03:37.369624 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:37.369469 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs" event={"ID":"9fc34b1b-977b-4209-9a27-8ae950b5e482","Type":"ContainerDied","Data":"7e0e8c68e6b4ff5929cb9f7d338dd3e83b6c6e76245dc7eae7afd4af8d8fabcb"} Aug 05 18:03:37.369624 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:37.369520 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e0e8c68e6b4ff5929cb9f7d338dd3e83b6c6e76245dc7eae7afd4af8d8fabcb" Aug 05 18:03:43.997239 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:43.997195 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs"] Aug 05 18:03:44.000219 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:44.000192 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-19f4b3-6nphs"] Aug 05 18:03:45.026859 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:45.026823 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fc34b1b-977b-4209-9a27-8ae950b5e482" path="/var/lib/kubelet/pods/9fc34b1b-977b-4209-9a27-8ae950b5e482/volumes" Aug 05 18:03:45.962853 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:03:45.962817 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:04:01.331731 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.331694 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx"] Aug 05 18:04:01.332713 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.332676 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" podUID="ea41d039-d0ee-4b61-aa0c-0d4a74127026" containerName="payload-processing" containerID="cri-o://9d944dd00db5046f1316891d53cb9dff569896093b4d7119ea574e492e62eb44" gracePeriod=30 Aug 05 18:04:01.511073 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.511006 2575 generic.go:356] "Generic (PLEG): container finished" podID="ea41d039-d0ee-4b61-aa0c-0d4a74127026" containerID="9d944dd00db5046f1316891d53cb9dff569896093b4d7119ea574e492e62eb44" exitCode=0 Aug 05 18:04:01.511254 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.511095 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" event={"ID":"ea41d039-d0ee-4b61-aa0c-0d4a74127026","Type":"ContainerDied","Data":"9d944dd00db5046f1316891d53cb9dff569896093b4d7119ea574e492e62eb44"} Aug 05 18:04:01.601720 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.601691 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:04:01.677596 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.677562 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea41d039-d0ee-4b61-aa0c-0d4a74127026-plugins-config-volume\") pod \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " Aug 05 18:04:01.677912 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.677890 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea41d039-d0ee-4b61-aa0c-0d4a74127026-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea41d039-d0ee-4b61-aa0c-0d4a74127026" (UID: "ea41d039-d0ee-4b61-aa0c-0d4a74127026"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:04:01.778347 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.778317 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j9d2\" (UniqueName: \"kubernetes.io/projected/ea41d039-d0ee-4b61-aa0c-0d4a74127026-kube-api-access-6j9d2\") pod \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\" (UID: \"ea41d039-d0ee-4b61-aa0c-0d4a74127026\") " Aug 05 18:04:01.778508 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.778482 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea41d039-d0ee-4b61-aa0c-0d4a74127026-plugins-config-volume\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:01.780404 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.780380 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea41d039-d0ee-4b61-aa0c-0d4a74127026-kube-api-access-6j9d2" (OuterVolumeSpecName: "kube-api-access-6j9d2") pod "ea41d039-d0ee-4b61-aa0c-0d4a74127026" (UID: "ea41d039-d0ee-4b61-aa0c-0d4a74127026"). InnerVolumeSpecName "kube-api-access-6j9d2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:04:01.879383 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:01.879299 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j9d2\" (UniqueName: \"kubernetes.io/projected/ea41d039-d0ee-4b61-aa0c-0d4a74127026-kube-api-access-6j9d2\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:02.517901 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:02.517872 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" Aug 05 18:04:02.517901 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:02.517883 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx" event={"ID":"ea41d039-d0ee-4b61-aa0c-0d4a74127026","Type":"ContainerDied","Data":"5a1ea72f16f50086078570adfb550df00334d820f0c41ea9cadfa9b52f89ffed"} Aug 05 18:04:02.518418 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:02.517944 2575 scope.go:117] "RemoveContainer" containerID="9d944dd00db5046f1316891d53cb9dff569896093b4d7119ea574e492e62eb44" Aug 05 18:04:02.542473 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:02.542448 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx"] Aug 05 18:04:02.549251 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:02.549230 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-76d454-59bff8b8dd-lnbsx"] Aug 05 18:04:03.026290 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:03.026257 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea41d039-d0ee-4b61-aa0c-0d4a74127026" path="/var/lib/kubelet/pods/ea41d039-d0ee-4b61-aa0c-0d4a74127026/volumes" Aug 05 18:04:03.366098 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:03.366048 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:04:08.047148 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:08.047115 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf"] Aug 05 18:04:08.047739 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:08.047321 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" containerID="cri-o://341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff" gracePeriod=30 Aug 05 18:04:13.303199 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.303162 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:04:13.386157 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386083 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5llgj\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-kube-api-access-5llgj\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386312 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386172 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-token\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386312 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386209 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-envoy\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386312 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386232 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-podinfo\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386312 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386259 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-credential-socket\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386312 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386286 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-certs\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386611 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386334 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d93bcf-992d-40bd-95b8-c0e38724724f-istiod-ca-cert\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386611 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386367 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-data\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386611 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386393 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-socket\") pod \"44d93bcf-992d-40bd-95b8-c0e38724724f\" (UID: \"44d93bcf-992d-40bd-95b8-c0e38724724f\") " Aug 05 18:04:13.386784 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386594 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:04:13.386784 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386655 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:04:13.386897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386801 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-certs\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.386897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386824 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-credential-socket\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.386897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386860 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:04:13.386897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386873 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-data" (OuterVolumeSpecName: "istio-data") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:04:13.387073 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.386904 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44d93bcf-992d-40bd-95b8-c0e38724724f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:04:13.388422 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.388395 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:04:13.388422 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.388406 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-kube-api-access-5llgj" (OuterVolumeSpecName: "kube-api-access-5llgj") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "kube-api-access-5llgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:04:13.388618 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.388494 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-token" (OuterVolumeSpecName: "istio-token") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:04:13.388618 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.388601 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44d93bcf-992d-40bd-95b8-c0e38724724f" (UID: "44d93bcf-992d-40bd-95b8-c0e38724724f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 05 18:04:13.480627 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.480591 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c"] Aug 05 18:04:13.480976 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.480963 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea41d039-d0ee-4b61-aa0c-0d4a74127026" containerName="payload-processing" Aug 05 18:04:13.481031 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.480977 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea41d039-d0ee-4b61-aa0c-0d4a74127026" containerName="payload-processing" Aug 05 18:04:13.481031 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.480994 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" Aug 05 18:04:13.481031 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.480999 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" Aug 05 18:04:13.481031 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481013 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fc34b1b-977b-4209-9a27-8ae950b5e482" containerName="revoke-keys" Aug 05 18:04:13.481031 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481019 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fc34b1b-977b-4209-9a27-8ae950b5e482" containerName="revoke-keys" Aug 05 18:04:13.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481034 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" Aug 05 18:04:13.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481039 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" Aug 05 18:04:13.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481121 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea41d039-d0ee-4b61-aa0c-0d4a74127026" containerName="payload-processing" Aug 05 18:04:13.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481133 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" Aug 05 18:04:13.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481139 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerName="istio-proxy" Aug 05 18:04:13.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.481149 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fc34b1b-977b-4209-9a27-8ae950b5e482" containerName="revoke-keys" Aug 05 18:04:13.484459 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.484441 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.486876 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.486848 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d5bdd1-dockercfg-ckq9s\"" Aug 05 18:04:13.487013 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.486851 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d5bdd1\"" Aug 05 18:04:13.487413 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487387 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44d93bcf-992d-40bd-95b8-c0e38724724f-istiod-ca-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.487413 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487409 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-data\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.487413 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487418 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-workload-socket\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.487594 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487427 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5llgj\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-kube-api-access-5llgj\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.487594 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487437 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-token\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.487594 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487446 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-envoy\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.487594 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.487454 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44d93bcf-992d-40bd-95b8-c0e38724724f-istio-podinfo\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:04:13.491479 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.491454 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c"] Aug 05 18:04:13.587359 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.587325 2575 generic.go:356] "Generic (PLEG): container finished" podID="44d93bcf-992d-40bd-95b8-c0e38724724f" containerID="341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff" exitCode=0 Aug 05 18:04:13.587530 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.587386 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" Aug 05 18:04:13.587530 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.587409 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" event={"ID":"44d93bcf-992d-40bd-95b8-c0e38724724f","Type":"ContainerDied","Data":"341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff"} Aug 05 18:04:13.587530 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.587462 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf" event={"ID":"44d93bcf-992d-40bd-95b8-c0e38724724f","Type":"ContainerDied","Data":"4be51d04a4b6a4d326424326f7ca9a77e5a41726bc4c6791aaae9303f87b74a3"} Aug 05 18:04:13.587530 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.587486 2575 scope.go:117] "RemoveContainer" containerID="341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff" Aug 05 18:04:13.587920 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.587891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwtf9\" (UniqueName: \"kubernetes.io/projected/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-kube-api-access-xwtf9\") pod \"payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.588076 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.588035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.597213 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.597197 2575 scope.go:117] "RemoveContainer" containerID="69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4" Aug 05 18:04:13.610358 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.610198 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf"] Aug 05 18:04:13.610631 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.610592 2575 scope.go:117] "RemoveContainer" containerID="341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff" Aug 05 18:04:13.610907 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:04:13.610887 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff\": container with ID starting with 341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff not found: ID does not exist" containerID="341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff" Aug 05 18:04:13.610973 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.610914 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff"} err="failed to get container status \"341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff\": rpc error: code = NotFound desc = could not find container \"341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff\": container with ID starting with 341fb07795a3f43c9150593299df70ccb194dc2746d08bc2422525b1b0ff28ff not found: ID does not exist" Aug 05 18:04:13.610973 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.610931 2575 scope.go:117] "RemoveContainer" containerID="69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4" Aug 05 18:04:13.611211 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:04:13.611193 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4\": container with ID starting with 69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4 not found: ID does not exist" containerID="69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4" Aug 05 18:04:13.611277 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.611216 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4"} err="failed to get container status \"69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4\": rpc error: code = NotFound desc = could not find container \"69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4\": container with ID starting with 69dd5dd29fa158462631e534d8fbcecae52cac74ea70e1db1b92e5ab102ac0d4 not found: ID does not exist" Aug 05 18:04:13.612600 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.612575 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-76d454-openshift-default-88b588864-kmfdf"] Aug 05 18:04:13.689303 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.689234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.689303 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.689296 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwtf9\" (UniqueName: \"kubernetes.io/projected/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-kube-api-access-xwtf9\") pod \"payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.689876 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.689857 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.696739 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.696712 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwtf9\" (UniqueName: \"kubernetes.io/projected/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-kube-api-access-xwtf9\") pod \"payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.797357 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.797324 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:13.937628 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:13.937600 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c"] Aug 05 18:04:13.939454 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:04:13.939390 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97df614a_3762_42e9_ac7a_03c3ec6eb3d6.slice/crio-bc27b835e5a6c8af221ffb7095117ca80720cc1007449f01f9daee8d539994b0 WatchSource:0}: Error finding container bc27b835e5a6c8af221ffb7095117ca80720cc1007449f01f9daee8d539994b0: Status 404 returned error can't find the container with id bc27b835e5a6c8af221ffb7095117ca80720cc1007449f01f9daee8d539994b0 Aug 05 18:04:14.595248 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:14.595209 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" event={"ID":"97df614a-3762-42e9-ac7a-03c3ec6eb3d6","Type":"ContainerStarted","Data":"2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5"} Aug 05 18:04:14.595699 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:14.595283 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:14.595699 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:14.595306 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" event={"ID":"97df614a-3762-42e9-ac7a-03c3ec6eb3d6","Type":"ContainerStarted","Data":"bc27b835e5a6c8af221ffb7095117ca80720cc1007449f01f9daee8d539994b0"} Aug 05 18:04:14.611601 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:14.611542 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" podStartSLOduration=1.6115265760000002 podStartE2EDuration="1.611526576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:14.609616248 +0000 UTC m=+2692.010845063" watchObservedRunningTime="2026-08-05 18:04:14.611526576 +0000 UTC m=+2692.012755390" Aug 05 18:04:15.027389 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:15.027267 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44d93bcf-992d-40bd-95b8-c0e38724724f" path="/var/lib/kubelet/pods/44d93bcf-992d-40bd-95b8-c0e38724724f/volumes" Aug 05 18:04:15.371110 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:15.371067 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:04:23.288904 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:23.288874 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:04:23.294844 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:23.294823 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:04:23.307344 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:23.307323 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:04:23.312667 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:23.312649 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:04:25.604046 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:25.604014 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:04:41.921124 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.921081 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq"] Aug 05 18:04:41.924647 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.924627 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:41.927003 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.926974 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-adc09f-serving-cert\"" Aug 05 18:04:41.927147 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.927016 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 18:04:41.927457 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.927421 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\"" Aug 05 18:04:41.927457 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.927450 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 05 18:04:41.936565 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:41.936529 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq"] Aug 05 18:04:42.055074 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.055024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-maas-api-tls\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.055261 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.055087 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k8x6\" (UniqueName: \"kubernetes.io/projected/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-kube-api-access-4k8x6\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.055261 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.055198 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-openshift-service-ca\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.156133 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.156090 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-maas-api-tls\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.156339 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.156169 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k8x6\" (UniqueName: \"kubernetes.io/projected/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-kube-api-access-4k8x6\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.156339 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.156326 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-openshift-service-ca\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.157252 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.157226 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-openshift-service-ca\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.158751 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.158729 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-maas-api-tls\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.164370 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.164352 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k8x6\" (UniqueName: \"kubernetes.io/projected/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-kube-api-access-4k8x6\") pod \"maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.239019 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.238942 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:42.377977 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.377941 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq"] Aug 05 18:04:42.379550 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:04:42.379523 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bb0a20_37ed_4772_92cc_4af5e60e8d8b.slice/crio-5d5018dd38dc4ae0c9462012eb40fb63b5cc8c51f708ab0c0d4c582015bb96fc WatchSource:0}: Error finding container 5d5018dd38dc4ae0c9462012eb40fb63b5cc8c51f708ab0c0d4c582015bb96fc: Status 404 returned error can't find the container with id 5d5018dd38dc4ae0c9462012eb40fb63b5cc8c51f708ab0c0d4c582015bb96fc Aug 05 18:04:42.381148 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.381124 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 18:04:42.753558 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:42.753521 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" event={"ID":"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b","Type":"ContainerStarted","Data":"5d5018dd38dc4ae0c9462012eb40fb63b5cc8c51f708ab0c0d4c582015bb96fc"} Aug 05 18:04:45.773730 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:45.773675 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" event={"ID":"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b","Type":"ContainerStarted","Data":"b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352"} Aug 05 18:04:45.774236 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:45.773748 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:04:45.791360 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:45.791302 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" podStartSLOduration=2.290200807 podStartE2EDuration="4.791287s" totalImagesPullingTime="2.501086193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:04:45.790170166 +0000 UTC m=+2723.191398984" watchObservedRunningTime="2026-08-05 18:04:45.791287 +0000 UTC m=+2723.192515815" Aug 05 18:04:46.160893 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:46.160855 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:04:51.785594 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:04:51.785563 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:05:08.846122 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.846090 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj"] Aug 05 18:05:08.849934 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.849919 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:08.852227 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.852018 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:05:08.855788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.855756 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj"] Aug 05 18:05:08.892101 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.892070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfbeedca-46d8-4632-b4d6-b56c99ff01a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj\" (UID: \"dfbeedca-46d8-4632-b4d6-b56c99ff01a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:08.993427 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.993377 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfbeedca-46d8-4632-b4d6-b56c99ff01a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj\" (UID: \"dfbeedca-46d8-4632-b4d6-b56c99ff01a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:08.994080 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:08.994031 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfbeedca-46d8-4632-b4d6-b56c99ff01a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj\" (UID: \"dfbeedca-46d8-4632-b4d6-b56c99ff01a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:09.162648 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:09.162566 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:09.293796 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:09.293768 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj"] Aug 05 18:05:09.295397 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:05:09.295367 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfbeedca_46d8_4632_b4d6_b56c99ff01a2.slice/crio-aad8f90262a9c9b6d40065c1abe51e6897f75e49a6b095ec1f21f61bfd81afde WatchSource:0}: Error finding container aad8f90262a9c9b6d40065c1abe51e6897f75e49a6b095ec1f21f61bfd81afde: Status 404 returned error can't find the container with id aad8f90262a9c9b6d40065c1abe51e6897f75e49a6b095ec1f21f61bfd81afde Aug 05 18:05:09.920476 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:09.920443 2575 generic.go:356] "Generic (PLEG): container finished" podID="dfbeedca-46d8-4632-b4d6-b56c99ff01a2" containerID="5d2571ebf5d82496d902230f5d99f0c8adde03aa3c45576c5732521d7cb6f188" exitCode=0 Aug 05 18:05:09.920831 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:09.920524 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" event={"ID":"dfbeedca-46d8-4632-b4d6-b56c99ff01a2","Type":"ContainerDied","Data":"5d2571ebf5d82496d902230f5d99f0c8adde03aa3c45576c5732521d7cb6f188"} Aug 05 18:05:09.920831 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:09.920583 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" event={"ID":"dfbeedca-46d8-4632-b4d6-b56c99ff01a2","Type":"ContainerStarted","Data":"aad8f90262a9c9b6d40065c1abe51e6897f75e49a6b095ec1f21f61bfd81afde"} Aug 05 18:05:11.055506 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.055483 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:11.110367 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.110334 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfbeedca-46d8-4632-b4d6-b56c99ff01a2-maas-api-service-ca\") pod \"dfbeedca-46d8-4632-b4d6-b56c99ff01a2\" (UID: \"dfbeedca-46d8-4632-b4d6-b56c99ff01a2\") " Aug 05 18:05:11.110689 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.110665 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfbeedca-46d8-4632-b4d6-b56c99ff01a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dfbeedca-46d8-4632-b4d6-b56c99ff01a2" (UID: "dfbeedca-46d8-4632-b4d6-b56c99ff01a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:05:11.211758 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.211663 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfbeedca-46d8-4632-b4d6-b56c99ff01a2-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:11.933504 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.933471 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" Aug 05 18:05:11.933749 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.933516 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj" event={"ID":"dfbeedca-46d8-4632-b4d6-b56c99ff01a2","Type":"ContainerDied","Data":"aad8f90262a9c9b6d40065c1abe51e6897f75e49a6b095ec1f21f61bfd81afde"} Aug 05 18:05:11.933749 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:11.933566 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aad8f90262a9c9b6d40065c1abe51e6897f75e49a6b095ec1f21f61bfd81afde" Aug 05 18:05:23.388950 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:23.388916 2575 scope.go:117] "RemoveContainer" containerID="578f96be25e6c3d757619323784e9ef4296653ce6b1de650e08df1a00b319991" Aug 05 18:05:24.572042 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.572003 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c"] Aug 05 18:05:24.572548 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.572260 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" podUID="97df614a-3762-42e9-ac7a-03c3ec6eb3d6" containerName="payload-processing" containerID="cri-o://2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5" gracePeriod=30 Aug 05 18:05:24.706558 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.706525 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj"] Aug 05 18:05:24.708852 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.708825 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d5bdd1-wxzqj"] Aug 05 18:05:24.827095 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.827022 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:05:24.940993 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.940956 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwtf9\" (UniqueName: \"kubernetes.io/projected/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-kube-api-access-xwtf9\") pod \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " Aug 05 18:05:24.940993 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.940993 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-plugins-config-volume\") pod \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\" (UID: \"97df614a-3762-42e9-ac7a-03c3ec6eb3d6\") " Aug 05 18:05:24.941433 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.941406 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97df614a-3762-42e9-ac7a-03c3ec6eb3d6" (UID: "97df614a-3762-42e9-ac7a-03c3ec6eb3d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:05:24.942972 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:24.942950 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-kube-api-access-xwtf9" (OuterVolumeSpecName: "kube-api-access-xwtf9") pod "97df614a-3762-42e9-ac7a-03c3ec6eb3d6" (UID: "97df614a-3762-42e9-ac7a-03c3ec6eb3d6"). InnerVolumeSpecName "kube-api-access-xwtf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:05:25.014344 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.014310 2575 generic.go:356] "Generic (PLEG): container finished" podID="97df614a-3762-42e9-ac7a-03c3ec6eb3d6" containerID="2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5" exitCode=0 Aug 05 18:05:25.014539 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.014360 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" Aug 05 18:05:25.014539 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.014391 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" event={"ID":"97df614a-3762-42e9-ac7a-03c3ec6eb3d6","Type":"ContainerDied","Data":"2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5"} Aug 05 18:05:25.014539 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.014442 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c" event={"ID":"97df614a-3762-42e9-ac7a-03c3ec6eb3d6","Type":"ContainerDied","Data":"bc27b835e5a6c8af221ffb7095117ca80720cc1007449f01f9daee8d539994b0"} Aug 05 18:05:25.014539 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.014464 2575 scope.go:117] "RemoveContainer" containerID="2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5" Aug 05 18:05:25.023631 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.023613 2575 scope.go:117] "RemoveContainer" containerID="2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5" Aug 05 18:05:25.023902 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:05:25.023883 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5\": container with ID starting with 2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5 not found: ID does not exist" containerID="2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5" Aug 05 18:05:25.023966 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.023911 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5"} err="failed to get container status \"2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5\": rpc error: code = NotFound desc = could not find container \"2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5\": container with ID starting with 2774809183ab9da7226d8eb0e198af8296c1ac76fda0927885c222f2d0db8ec5 not found: ID does not exist" Aug 05 18:05:25.027355 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.027334 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfbeedca-46d8-4632-b4d6-b56c99ff01a2" path="/var/lib/kubelet/pods/dfbeedca-46d8-4632-b4d6-b56c99ff01a2/volumes" Aug 05 18:05:25.035892 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.035870 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c"] Aug 05 18:05:25.041248 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.041223 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d5bdd1-54449bd5f6-vns7c"] Aug 05 18:05:25.041606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.041589 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwtf9\" (UniqueName: \"kubernetes.io/projected/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-kube-api-access-xwtf9\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:25.041653 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:25.041610 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97df614a-3762-42e9-ac7a-03c3ec6eb3d6-plugins-config-volume\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:26.970402 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:26.970360 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:05:27.027907 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:27.027872 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97df614a-3762-42e9-ac7a-03c3ec6eb3d6" path="/var/lib/kubelet/pods/97df614a-3762-42e9-ac7a-03c3ec6eb3d6/volumes" Aug 05 18:05:31.290904 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.290868 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh"] Aug 05 18:05:31.291343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.291310 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97df614a-3762-42e9-ac7a-03c3ec6eb3d6" containerName="payload-processing" Aug 05 18:05:31.291343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.291324 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="97df614a-3762-42e9-ac7a-03c3ec6eb3d6" containerName="payload-processing" Aug 05 18:05:31.291343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.291337 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfbeedca-46d8-4632-b4d6-b56c99ff01a2" containerName="revoke-keys" Aug 05 18:05:31.291343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.291343 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfbeedca-46d8-4632-b4d6-b56c99ff01a2" containerName="revoke-keys" Aug 05 18:05:31.291476 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.291414 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="97df614a-3762-42e9-ac7a-03c3ec6eb3d6" containerName="payload-processing" Aug 05 18:05:31.291476 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.291421 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfbeedca-46d8-4632-b4d6-b56c99ff01a2" containerName="revoke-keys" Aug 05 18:05:31.295794 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.295778 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:31.297787 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.297766 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:05:31.301829 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.301805 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh"] Aug 05 18:05:31.398914 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.398880 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2283b631-07a9-4307-9a9f-4262253078b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh\" (UID: \"2283b631-07a9-4307-9a9f-4262253078b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:31.500112 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.500046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2283b631-07a9-4307-9a9f-4262253078b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh\" (UID: \"2283b631-07a9-4307-9a9f-4262253078b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:31.500658 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.500639 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2283b631-07a9-4307-9a9f-4262253078b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh\" (UID: \"2283b631-07a9-4307-9a9f-4262253078b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:31.608679 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.608636 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:31.744510 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:31.744485 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh"] Aug 05 18:05:31.747956 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:05:31.747916 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2283b631_07a9_4307_9a9f_4262253078b0.slice/crio-68b0802ac16961a12602abfc84c04ececfe265ee73ecb7fe26a932ba43dfa59a WatchSource:0}: Error finding container 68b0802ac16961a12602abfc84c04ececfe265ee73ecb7fe26a932ba43dfa59a: Status 404 returned error can't find the container with id 68b0802ac16961a12602abfc84c04ececfe265ee73ecb7fe26a932ba43dfa59a Aug 05 18:05:32.055329 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:32.055292 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" event={"ID":"2283b631-07a9-4307-9a9f-4262253078b0","Type":"ContainerStarted","Data":"538ed069da8ac8d387f5fafde873c597fbc3160a57535a91acfb23d38681235e"} Aug 05 18:05:32.055426 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:32.055393 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" event={"ID":"2283b631-07a9-4307-9a9f-4262253078b0","Type":"ContainerStarted","Data":"68b0802ac16961a12602abfc84c04ececfe265ee73ecb7fe26a932ba43dfa59a"} Aug 05 18:05:32.069640 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:32.069597 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" podStartSLOduration=1.069582481 podStartE2EDuration="1.069582481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:05:32.06766767 +0000 UTC m=+2769.468896486" watchObservedRunningTime="2026-08-05 18:05:32.069582481 +0000 UTC m=+2769.470811294" Aug 05 18:05:33.062420 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:33.062384 2575 generic.go:356] "Generic (PLEG): container finished" podID="2283b631-07a9-4307-9a9f-4262253078b0" containerID="538ed069da8ac8d387f5fafde873c597fbc3160a57535a91acfb23d38681235e" exitCode=0 Aug 05 18:05:33.062807 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:33.062472 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" event={"ID":"2283b631-07a9-4307-9a9f-4262253078b0","Type":"ContainerDied","Data":"538ed069da8ac8d387f5fafde873c597fbc3160a57535a91acfb23d38681235e"} Aug 05 18:05:34.199773 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:34.199743 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:34.326798 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:34.326706 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2283b631-07a9-4307-9a9f-4262253078b0-maas-api-service-ca\") pod \"2283b631-07a9-4307-9a9f-4262253078b0\" (UID: \"2283b631-07a9-4307-9a9f-4262253078b0\") " Aug 05 18:05:34.327157 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:34.327131 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2283b631-07a9-4307-9a9f-4262253078b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2283b631-07a9-4307-9a9f-4262253078b0" (UID: "2283b631-07a9-4307-9a9f-4262253078b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:05:34.427825 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:34.427787 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2283b631-07a9-4307-9a9f-4262253078b0-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:35.074614 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:35.074578 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" Aug 05 18:05:35.074831 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:35.074609 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh" event={"ID":"2283b631-07a9-4307-9a9f-4262253078b0","Type":"ContainerDied","Data":"68b0802ac16961a12602abfc84c04ececfe265ee73ecb7fe26a932ba43dfa59a"} Aug 05 18:05:35.074831 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:35.074658 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68b0802ac16961a12602abfc84c04ececfe265ee73ecb7fe26a932ba43dfa59a" Aug 05 18:05:46.744225 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:46.744139 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq"] Aug 05 18:05:46.744717 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:46.744378 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" podUID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" containerName="maas-api" containerID="cri-o://b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352" gracePeriod=30 Aug 05 18:05:46.780144 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:46.780105 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" podUID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.60:8443/health\": dial tcp 10.132.0.60:8443: connect: connection refused" Aug 05 18:05:46.956610 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:46.956569 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh"] Aug 05 18:05:46.959927 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:46.959893 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-adc09f-v5zmh"] Aug 05 18:05:47.000432 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.000368 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:05:47.028368 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.028337 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2283b631-07a9-4307-9a9f-4262253078b0" path="/var/lib/kubelet/pods/2283b631-07a9-4307-9a9f-4262253078b0/volumes" Aug 05 18:05:47.044700 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.044668 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-maas-api-tls\") pod \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " Aug 05 18:05:47.044868 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.044754 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4k8x6\" (UniqueName: \"kubernetes.io/projected/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-kube-api-access-4k8x6\") pod \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " Aug 05 18:05:47.044868 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.044811 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-openshift-service-ca\") pod \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\" (UID: \"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b\") " Aug 05 18:05:47.045217 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.045192 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" (UID: "f0bb0a20-37ed-4772-92cc-4af5e60e8d8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:05:47.046997 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.046969 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" (UID: "f0bb0a20-37ed-4772-92cc-4af5e60e8d8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 18:05:47.046997 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.046984 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-kube-api-access-4k8x6" (OuterVolumeSpecName: "kube-api-access-4k8x6") pod "f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" (UID: "f0bb0a20-37ed-4772-92cc-4af5e60e8d8b"). InnerVolumeSpecName "kube-api-access-4k8x6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:05:47.145917 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.145884 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4k8x6\" (UniqueName: \"kubernetes.io/projected/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-kube-api-access-4k8x6\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:47.145917 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.145917 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-openshift-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:47.146159 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.145932 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b-maas-api-tls\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:05:47.149814 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.149787 2575 generic.go:356] "Generic (PLEG): container finished" podID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" containerID="b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352" exitCode=0 Aug 05 18:05:47.149950 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.149837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" event={"ID":"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b","Type":"ContainerDied","Data":"b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352"} Aug 05 18:05:47.149950 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.149873 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" event={"ID":"f0bb0a20-37ed-4772-92cc-4af5e60e8d8b","Type":"ContainerDied","Data":"5d5018dd38dc4ae0c9462012eb40fb63b5cc8c51f708ab0c0d4c582015bb96fc"} Aug 05 18:05:47.149950 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.149899 2575 scope.go:117] "RemoveContainer" containerID="b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352" Aug 05 18:05:47.149950 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.149837 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq" Aug 05 18:05:47.159674 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.159656 2575 scope.go:117] "RemoveContainer" containerID="b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352" Aug 05 18:05:47.159942 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:05:47.159907 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352\": container with ID starting with b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352 not found: ID does not exist" containerID="b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352" Aug 05 18:05:47.160024 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.159946 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352"} err="failed to get container status \"b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352\": rpc error: code = NotFound desc = could not find container \"b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352\": container with ID starting with b7e5315f02d7a0f1921eaec4b77ee1f7fc982126929f023eaf319bf4069b7352 not found: ID does not exist" Aug 05 18:05:47.175178 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.175155 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq"] Aug 05 18:05:47.181239 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:47.181218 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-adc09f-864c8bc66-gg9xq"] Aug 05 18:05:48.961356 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:48.961315 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:05:49.027833 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:05:49.027798 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" path="/var/lib/kubelet/pods/f0bb0a20-37ed-4772-92cc-4af5e60e8d8b/volumes" Aug 05 18:06:01.267764 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:01.267730 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:06:22.773039 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773005 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9"] Aug 05 18:06:22.773462 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773450 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" containerName="maas-api" Aug 05 18:06:22.773462 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773463 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" containerName="maas-api" Aug 05 18:06:22.773541 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773473 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2283b631-07a9-4307-9a9f-4262253078b0" containerName="revoke-keys" Aug 05 18:06:22.773541 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773479 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2283b631-07a9-4307-9a9f-4262253078b0" containerName="revoke-keys" Aug 05 18:06:22.773605 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773556 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2283b631-07a9-4307-9a9f-4262253078b0" containerName="revoke-keys" Aug 05 18:06:22.773605 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.773568 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0bb0a20-37ed-4772-92cc-4af5e60e8d8b" containerName="maas-api" Aug 05 18:06:22.775531 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.775513 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.777726 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.777696 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-30613e-openshift-default-dockercfg-tlf9t\"" Aug 05 18:06:22.779089 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.779038 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 05 18:06:22.788833 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.788807 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9"] Aug 05 18:06:22.870828 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.870789 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-data\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.870828 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.870835 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d703899-6f4a-4711-ab32-7384a51cb8ca-istiod-ca-cert\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.870858 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-envoy\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.870941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-podinfo\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.871076 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-credential-socket\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.871117 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-socket\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.871152 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-certs\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.871212 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8btn5\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-kube-api-access-8btn5\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.871417 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.871244 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-token\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.971774 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971740 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8btn5\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-kube-api-access-8btn5\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.971774 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-token\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971799 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-data\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d703899-6f4a-4711-ab32-7384a51cb8ca-istiod-ca-cert\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971841 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-envoy\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971872 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-podinfo\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-credential-socket\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.971968 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-socket\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.972005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-certs\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972409 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.972359 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-data\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972468 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.972408 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-credential-socket\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972468 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.972447 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-socket\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.972590 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.972475 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-certs\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.974001 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.973981 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 05 18:06:22.974270 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.974253 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-envoy\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.974490 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.974472 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-podinfo\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.978995 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.978971 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-token\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.979214 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.979192 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8btn5\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-kube-api-access-8btn5\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:22.983364 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:22.983341 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d703899-6f4a-4711-ab32-7384a51cb8ca-istiod-ca-cert\") pod \"e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:23.088517 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.088488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-30613e-openshift-default-dockercfg-tlf9t\"" Aug 05 18:06:23.096244 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.096215 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:23.233987 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.233962 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9"] Aug 05 18:06:23.235178 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:06:23.235153 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d703899_6f4a_4711_ab32_7384a51cb8ca.slice/crio-ef3ac13d809c2074db942737a90872264fc7ff9b31daca2f08827a3364952827 WatchSource:0}: Error finding container ef3ac13d809c2074db942737a90872264fc7ff9b31daca2f08827a3364952827: Status 404 returned error can't find the container with id ef3ac13d809c2074db942737a90872264fc7ff9b31daca2f08827a3364952827 Aug 05 18:06:23.237290 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.237257 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:06:23.237383 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.237321 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:06:23.237383 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.237353 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:06:23.354695 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.354599 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" event={"ID":"4d703899-6f4a-4711-ab32-7384a51cb8ca","Type":"ContainerStarted","Data":"daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc"} Aug 05 18:06:23.354695 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.354654 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" event={"ID":"4d703899-6f4a-4711-ab32-7384a51cb8ca","Type":"ContainerStarted","Data":"ef3ac13d809c2074db942737a90872264fc7ff9b31daca2f08827a3364952827"} Aug 05 18:06:23.377620 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:23.377565 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podStartSLOduration=1.3775493939999999 podStartE2EDuration="1.377549394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:06:23.375369725 +0000 UTC m=+2820.776598540" watchObservedRunningTime="2026-08-05 18:06:23.377549394 +0000 UTC m=+2820.778778209" Aug 05 18:06:24.096483 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:24.096442 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:24.101486 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:24.101460 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:24.360976 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:24.360873 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:24.361739 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:24.361716 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:29.664330 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:29.664292 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:06:39.638896 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:39.638849 2575 patch_prober.go:28] interesting pod/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:55902->10.132.0.63:15021: read: connection reset by peer" start-of-body= Aug 05 18:06:39.639557 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:39.638920 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:55902->10.132.0.63:15021: read: connection reset by peer" Aug 05 18:06:40.455800 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:40.455764 2575 generic.go:356] "Generic (PLEG): container finished" podID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerID="daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc" exitCode=0 Aug 05 18:06:40.455989 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:40.455834 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" event={"ID":"4d703899-6f4a-4711-ab32-7384a51cb8ca","Type":"ContainerDied","Data":"daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc"} Aug 05 18:06:40.456253 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:40.456236 2575 scope.go:117] "RemoveContainer" containerID="daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc" Aug 05 18:06:40.457224 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:40.457196 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:06:40.457315 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:40.457270 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:06:40.457315 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:40.457304 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 05 18:06:41.466306 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:41.466264 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" event={"ID":"4d703899-6f4a-4711-ab32-7384a51cb8ca","Type":"ContainerStarted","Data":"bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0"} Aug 05 18:06:42.096805 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:42.096766 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:42.097004 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:42.096982 2575 patch_prober.go:28] interesting pod/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 05 18:06:42.097080 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:42.097028 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 05 18:06:43.096719 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:43.096676 2575 patch_prober.go:28] interesting pod/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 05 18:06:43.097136 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:43.096750 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 05 18:06:44.096734 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:44.096697 2575 patch_prober.go:28] interesting pod/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 05 18:06:44.097159 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:44.096756 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 05 18:06:45.096825 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:45.096783 2575 patch_prober.go:28] interesting pod/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 05 18:06:45.097354 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:45.096853 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 05 18:06:46.096792 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:46.096758 2575 patch_prober.go:28] interesting pod/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 05 18:06:46.096972 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:46.096827 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 05 18:06:47.100783 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:47.100747 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:47.101300 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:47.101035 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:47.101837 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:47.101817 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:06:58.077586 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:06:58.077546 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:07:21.863450 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:07:21.863364 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:07:45.465355 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:07:45.465314 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:07:58.971156 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:07:58.971117 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:08:10.472857 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:08:10.472818 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:08:25.881208 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:08:25.881171 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:08:38.073739 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:08:38.073698 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:08:51.971932 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:08:51.971846 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:09:03.496913 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:03.496879 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:09:18.768742 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:18.768707 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:09:23.328511 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:23.328476 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:09:23.334341 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:23.334315 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:09:23.346967 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:23.346942 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:09:23.352464 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:23.352440 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:09:30.262961 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:30.262924 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:09:45.074888 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:45.074852 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:09:56.662914 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:09:56.662871 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:10:11.659012 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:11.658976 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:10:20.464752 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.464672 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl"] Aug 05 18:10:20.467116 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.467098 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.469360 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.469334 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"openshift-service-ca.crt\"" Aug 05 18:10:20.469778 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.469757 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"kube-root-ca.crt\"" Aug 05 18:10:20.469865 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.469785 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"sub-test-model-3f8e0d-kserve-self-signed-certs\"" Aug 05 18:10:20.469865 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.469843 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3f8e0d\"/\"default-dockercfg-xpvxf\"" Aug 05 18:10:20.477272 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.477251 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl"] Aug 05 18:10:20.570341 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.570309 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-home\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.570518 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.570357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-dshm\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.570518 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.570476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-model-cache\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.570610 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.570522 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tls-certs\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.570655 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.570602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dglj5\" (UniqueName: \"kubernetes.io/projected/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-kube-api-access-dglj5\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.570655 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.570639 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tmp-dir\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.671542 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.671499 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tls-certs\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.671706 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.671566 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dglj5\" (UniqueName: \"kubernetes.io/projected/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-kube-api-access-dglj5\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.671706 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.671600 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tmp-dir\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.671706 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.671635 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-home\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.671706 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.671672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-dshm\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.671857 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.671746 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-model-cache\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.672045 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.672012 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tmp-dir\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.672189 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.672085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-home\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.672258 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.672209 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-model-cache\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.673874 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.673845 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-dshm\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.674087 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.674072 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tls-certs\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.679373 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.679352 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dglj5\" (UniqueName: \"kubernetes.io/projected/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-kube-api-access-dglj5\") pod \"sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.777998 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.777888 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:20.912574 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.912551 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl"] Aug 05 18:10:20.914421 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:10:20.914395 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9cde7f7_ebdb_46d5_bbaf_963cfb227e57.slice/crio-e64d7f474cfbc7385008edef6cf6c958eed3a95016d87f2a73da398403a18865 WatchSource:0}: Error finding container e64d7f474cfbc7385008edef6cf6c958eed3a95016d87f2a73da398403a18865: Status 404 returned error can't find the container with id e64d7f474cfbc7385008edef6cf6c958eed3a95016d87f2a73da398403a18865 Aug 05 18:10:20.916343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:20.916327 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 18:10:21.744596 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:21.744554 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" event={"ID":"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57","Type":"ContainerStarted","Data":"e64d7f474cfbc7385008edef6cf6c958eed3a95016d87f2a73da398403a18865"} Aug 05 18:10:22.661258 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:22.661217 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:10:22.752255 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:22.752217 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" event={"ID":"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57","Type":"ContainerStarted","Data":"b07ca0c2e9ac162801eaf184933ca4c679a16b51afcf4a81a01598eadc7fc152"} Aug 05 18:10:22.770464 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:22.770410 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" podStartSLOduration=1.535191188 podStartE2EDuration="2.770394563s" totalImagesPullingTime="1.235203375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:10:22.768247954 +0000 UTC m=+3060.169476769" watchObservedRunningTime="2026-08-05 18:10:22.770394563 +0000 UTC m=+3060.171623376" Aug 05 18:10:23.457967 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:23.457935 2575 scope.go:117] "RemoveContainer" containerID="1cae8be741ee6236f3041975843d0187ce32b94bdb782a84bcf6a42b7b45f9af" Aug 05 18:10:30.778282 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:30.778222 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:30.778282 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:30.778294 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:30.790540 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:30.790504 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:30.806732 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:30.806702 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:10:48.663593 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:10:48.663559 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:11:23.480986 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:11:23.480951 2575 scope.go:117] "RemoveContainer" containerID="5d2571ebf5d82496d902230f5d99f0c8adde03aa3c45576c5732521d7cb6f188" Aug 05 18:11:23.865444 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:11:23.865406 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:11:36.479136 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:11:36.479095 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:11:48.172162 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:11:48.172078 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:11:52.106737 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:11:52.106697 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl"] Aug 05 18:11:58.161710 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:11:58.161667 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:12:07.127462 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.127418 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" podUID="e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" containerName="main" containerID="cri-o://b07ca0c2e9ac162801eaf184933ca4c679a16b51afcf4a81a01598eadc7fc152" gracePeriod=45 Aug 05 18:12:07.340163 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.340127 2575 generic.go:356] "Generic (PLEG): container finished" podID="e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" containerID="b07ca0c2e9ac162801eaf184933ca4c679a16b51afcf4a81a01598eadc7fc152" exitCode=0 Aug 05 18:12:07.340349 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.340218 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" event={"ID":"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57","Type":"ContainerDied","Data":"b07ca0c2e9ac162801eaf184933ca4c679a16b51afcf4a81a01598eadc7fc152"} Aug 05 18:12:07.375405 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.375376 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:12:07.377382 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377346 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-dshm\") pod \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " Aug 05 18:12:07.377551 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377399 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tmp-dir\") pod \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " Aug 05 18:12:07.377551 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377441 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dglj5\" (UniqueName: \"kubernetes.io/projected/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-kube-api-access-dglj5\") pod \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " Aug 05 18:12:07.377551 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377473 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tls-certs\") pod \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " Aug 05 18:12:07.377551 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377530 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-model-cache\") pod \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " Aug 05 18:12:07.377745 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377569 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-home\") pod \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\" (UID: \"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57\") " Aug 05 18:12:07.377808 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377765 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" (UID: "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:12:07.377808 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377777 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" (UID: "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:12:07.377917 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377884 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-home" (OuterVolumeSpecName: "home") pod "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" (UID: "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:12:07.378010 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.377991 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tmp-dir\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:12:07.378089 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.378018 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-model-cache\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:12:07.378089 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.378034 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-home\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:12:07.379706 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.379681 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" (UID: "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 18:12:07.379825 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.379798 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-dshm" (OuterVolumeSpecName: "dshm") pod "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" (UID: "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:12:07.379881 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.379819 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-kube-api-access-dglj5" (OuterVolumeSpecName: "kube-api-access-dglj5") pod "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" (UID: "e9cde7f7-ebdb-46d5-bbaf-963cfb227e57"). InnerVolumeSpecName "kube-api-access-dglj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:12:07.479018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.478976 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dglj5\" (UniqueName: \"kubernetes.io/projected/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-kube-api-access-dglj5\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:12:07.479018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.479011 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-tls-certs\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:12:07.479018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:07.479025 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57-dshm\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:12:08.347217 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:08.347184 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" Aug 05 18:12:08.347638 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:08.347184 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl" event={"ID":"e9cde7f7-ebdb-46d5-bbaf-963cfb227e57","Type":"ContainerDied","Data":"e64d7f474cfbc7385008edef6cf6c958eed3a95016d87f2a73da398403a18865"} Aug 05 18:12:08.347638 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:08.347315 2575 scope.go:117] "RemoveContainer" containerID="b07ca0c2e9ac162801eaf184933ca4c679a16b51afcf4a81a01598eadc7fc152" Aug 05 18:12:08.371665 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:08.371639 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl"] Aug 05 18:12:08.376819 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:08.376796 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3f8e0d/sub-test-model-3f8e0d-kserve-7b794cb96c-dcxgl"] Aug 05 18:12:09.026680 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:09.026644 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" path="/var/lib/kubelet/pods/e9cde7f7-ebdb-46d5-bbaf-963cfb227e57/volumes" Aug 05 18:12:10.769441 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:10.769403 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:12:23.502061 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:23.502013 2575 scope.go:117] "RemoveContainer" containerID="538ed069da8ac8d387f5fafde873c597fbc3160a57535a91acfb23d38681235e" Aug 05 18:12:29.861751 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:12:29.861710 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:13:05.267198 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:05.267158 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:13:13.676907 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:13.676867 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:13:29.661382 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:29.661341 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:13:36.952692 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.952659 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-140246292930224"] Aug 05 18:13:36.953232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.953060 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" containerName="main" Aug 05 18:13:36.953232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.953072 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" containerName="main" Aug 05 18:13:36.953232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.953159 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9cde7f7-ebdb-46d5-bbaf-963cfb227e57" containerName="main" Aug 05 18:13:36.956307 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.956289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:36.958356 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.958331 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jqqxp\"" Aug 05 18:13:36.962355 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:36.962333 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246292930224"] Aug 05 18:13:37.027757 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:37.027728 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dddqg\" (UniqueName: \"kubernetes.io/projected/fb6cc23c-0316-4453-8ad4-47a9d808f35b-kube-api-access-dddqg\") pod \"test-curl-7534-140246292930224\" (UID: \"fb6cc23c-0316-4453-8ad4-47a9d808f35b\") " pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:37.128760 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:37.128715 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dddqg\" (UniqueName: \"kubernetes.io/projected/fb6cc23c-0316-4453-8ad4-47a9d808f35b-kube-api-access-dddqg\") pod \"test-curl-7534-140246292930224\" (UID: \"fb6cc23c-0316-4453-8ad4-47a9d808f35b\") " pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:37.136456 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:37.136428 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dddqg\" (UniqueName: \"kubernetes.io/projected/fb6cc23c-0316-4453-8ad4-47a9d808f35b-kube-api-access-dddqg\") pod \"test-curl-7534-140246292930224\" (UID: \"fb6cc23c-0316-4453-8ad4-47a9d808f35b\") " pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:37.266784 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:37.266681 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:37.404723 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:37.404692 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246292930224"] Aug 05 18:13:37.406419 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:13:37.406394 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb6cc23c_0316_4453_8ad4_47a9d808f35b.slice/crio-36dee28cc11e23ee73b026253b7d5bc760b58ecc5284b90493281ce3542e9191 WatchSource:0}: Error finding container 36dee28cc11e23ee73b026253b7d5bc760b58ecc5284b90493281ce3542e9191: Status 404 returned error can't find the container with id 36dee28cc11e23ee73b026253b7d5bc760b58ecc5284b90493281ce3542e9191 Aug 05 18:13:37.860312 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:37.860276 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246292930224" event={"ID":"fb6cc23c-0316-4453-8ad4-47a9d808f35b","Type":"ContainerStarted","Data":"36dee28cc11e23ee73b026253b7d5bc760b58ecc5284b90493281ce3542e9191"} Aug 05 18:13:38.867443 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:38.867402 2575 generic.go:356] "Generic (PLEG): container finished" podID="fb6cc23c-0316-4453-8ad4-47a9d808f35b" containerID="3a128b98ead23a90c3e4975a13f15950211c1754094ab0094b93d62d9a69e41e" exitCode=0 Aug 05 18:13:38.867929 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:38.867439 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246292930224" event={"ID":"fb6cc23c-0316-4453-8ad4-47a9d808f35b","Type":"ContainerDied","Data":"3a128b98ead23a90c3e4975a13f15950211c1754094ab0094b93d62d9a69e41e"} Aug 05 18:13:40.011128 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.011096 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:40.042393 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.042357 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7534-140246292930224_fb6cc23c-0316-4453-8ad4-47a9d808f35b/test-curl-7534-140246292930224/0.log" Aug 05 18:13:40.058496 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.058458 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dddqg\" (UniqueName: \"kubernetes.io/projected/fb6cc23c-0316-4453-8ad4-47a9d808f35b-kube-api-access-dddqg\") pod \"fb6cc23c-0316-4453-8ad4-47a9d808f35b\" (UID: \"fb6cc23c-0316-4453-8ad4-47a9d808f35b\") " Aug 05 18:13:40.060729 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.060696 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6cc23c-0316-4453-8ad4-47a9d808f35b-kube-api-access-dddqg" (OuterVolumeSpecName: "kube-api-access-dddqg") pod "fb6cc23c-0316-4453-8ad4-47a9d808f35b" (UID: "fb6cc23c-0316-4453-8ad4-47a9d808f35b"). InnerVolumeSpecName "kube-api-access-dddqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:13:40.064830 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.064805 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-140246292930224"] Aug 05 18:13:40.071085 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.071045 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-140246292930224"] Aug 05 18:13:40.159667 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.159581 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dddqg\" (UniqueName: \"kubernetes.io/projected/fb6cc23c-0316-4453-8ad4-47a9d808f35b-kube-api-access-dddqg\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:13:40.214884 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.214844 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-140246292930096"] Aug 05 18:13:40.215324 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.215305 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb6cc23c-0316-4453-8ad4-47a9d808f35b" containerName="test-curl-7534-140246292930224" Aug 05 18:13:40.215438 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.215326 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6cc23c-0316-4453-8ad4-47a9d808f35b" containerName="test-curl-7534-140246292930224" Aug 05 18:13:40.215500 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.215458 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb6cc23c-0316-4453-8ad4-47a9d808f35b" containerName="test-curl-7534-140246292930224" Aug 05 18:13:40.218582 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.218562 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:40.221935 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.221912 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246292930096"] Aug 05 18:13:40.260741 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.260712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tlwp\" (UniqueName: \"kubernetes.io/projected/0ff6fd87-3e88-4053-bc59-5d809a167e77-kube-api-access-7tlwp\") pod \"test-curl-7534-140246292930096\" (UID: \"0ff6fd87-3e88-4053-bc59-5d809a167e77\") " pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:40.361206 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.361171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tlwp\" (UniqueName: \"kubernetes.io/projected/0ff6fd87-3e88-4053-bc59-5d809a167e77-kube-api-access-7tlwp\") pod \"test-curl-7534-140246292930096\" (UID: \"0ff6fd87-3e88-4053-bc59-5d809a167e77\") " pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:40.369405 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.369381 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tlwp\" (UniqueName: \"kubernetes.io/projected/0ff6fd87-3e88-4053-bc59-5d809a167e77-kube-api-access-7tlwp\") pod \"test-curl-7534-140246292930096\" (UID: \"0ff6fd87-3e88-4053-bc59-5d809a167e77\") " pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:40.529428 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.529344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:40.661293 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.661260 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246292930096"] Aug 05 18:13:40.661997 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:13:40.661974 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ff6fd87_3e88_4053_bc59_5d809a167e77.slice/crio-18aaa1d618c49f952e963af1e8bd9237456ba1d9eca3739b8efe8fffbd4294a0 WatchSource:0}: Error finding container 18aaa1d618c49f952e963af1e8bd9237456ba1d9eca3739b8efe8fffbd4294a0: Status 404 returned error can't find the container with id 18aaa1d618c49f952e963af1e8bd9237456ba1d9eca3739b8efe8fffbd4294a0 Aug 05 18:13:40.666238 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.666218 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:13:40.879671 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.879638 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:40.879671 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.879662 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36dee28cc11e23ee73b026253b7d5bc760b58ecc5284b90493281ce3542e9191" Aug 05 18:13:40.880960 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:40.880935 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246292930096" event={"ID":"0ff6fd87-3e88-4053-bc59-5d809a167e77","Type":"ContainerStarted","Data":"18aaa1d618c49f952e963af1e8bd9237456ba1d9eca3739b8efe8fffbd4294a0"} Aug 05 18:13:40.891187 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:13:40.891150 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-140246292930224\" is forbidden: User \"system:node:ip-10-0-143-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-58.ec2.internal' and this object" podUID="fb6cc23c-0316-4453-8ad4-47a9d808f35b" pod="opendatahub/test-curl-7534-140246292930224" Aug 05 18:13:41.026908 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:41.026875 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb6cc23c-0316-4453-8ad4-47a9d808f35b" path="/var/lib/kubelet/pods/fb6cc23c-0316-4453-8ad4-47a9d808f35b/volumes" Aug 05 18:13:41.891766 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:41.891732 2575 generic.go:356] "Generic (PLEG): container finished" podID="0ff6fd87-3e88-4053-bc59-5d809a167e77" containerID="66aa8ba580707041ec19fc892e1879369490e0d026b239c6455ebe056819eac7" exitCode=0 Aug 05 18:13:41.891954 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:41.891775 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246292930096" event={"ID":"0ff6fd87-3e88-4053-bc59-5d809a167e77","Type":"ContainerDied","Data":"66aa8ba580707041ec19fc892e1879369490e0d026b239c6455ebe056819eac7"} Aug 05 18:13:43.031413 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.031388 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:43.051699 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.051670 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7534-140246292930096_0ff6fd87-3e88-4053-bc59-5d809a167e77/test-curl-7534-140246292930096/0.log" Aug 05 18:13:43.074352 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.074305 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-140246292930096"] Aug 05 18:13:43.076634 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.076610 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-140246292930096"] Aug 05 18:13:43.086832 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.086810 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tlwp\" (UniqueName: \"kubernetes.io/projected/0ff6fd87-3e88-4053-bc59-5d809a167e77-kube-api-access-7tlwp\") pod \"0ff6fd87-3e88-4053-bc59-5d809a167e77\" (UID: \"0ff6fd87-3e88-4053-bc59-5d809a167e77\") " Aug 05 18:13:43.088960 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.088931 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ff6fd87-3e88-4053-bc59-5d809a167e77-kube-api-access-7tlwp" (OuterVolumeSpecName: "kube-api-access-7tlwp") pod "0ff6fd87-3e88-4053-bc59-5d809a167e77" (UID: "0ff6fd87-3e88-4053-bc59-5d809a167e77"). InnerVolumeSpecName "kube-api-access-7tlwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:13:43.188635 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.188555 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tlwp\" (UniqueName: \"kubernetes.io/projected/0ff6fd87-3e88-4053-bc59-5d809a167e77-kube-api-access-7tlwp\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:13:43.243432 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.243400 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:43.243802 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.243789 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ff6fd87-3e88-4053-bc59-5d809a167e77" containerName="test-curl-7534-140246292930096" Aug 05 18:13:43.243850 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.243803 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ff6fd87-3e88-4053-bc59-5d809a167e77" containerName="test-curl-7534-140246292930096" Aug 05 18:13:43.243902 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.243875 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ff6fd87-3e88-4053-bc59-5d809a167e77" containerName="test-curl-7534-140246292930096" Aug 05 18:13:43.247035 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.247015 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:43.252525 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.251999 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:43.390458 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.390421 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrrsp\" (UniqueName: \"kubernetes.io/projected/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd-kube-api-access-qrrsp\") pod \"test-curl-7534-140246293067792\" (UID: \"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd\") " pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:43.491944 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.491849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrrsp\" (UniqueName: \"kubernetes.io/projected/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd-kube-api-access-qrrsp\") pod \"test-curl-7534-140246293067792\" (UID: \"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd\") " pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:43.500788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.500754 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrrsp\" (UniqueName: \"kubernetes.io/projected/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd-kube-api-access-qrrsp\") pod \"test-curl-7534-140246293067792\" (UID: \"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd\") " pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:43.559483 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.559448 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:43.693872 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.693841 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:43.695925 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:13:43.695896 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c8cec83_24ed_46c0_8815_00ad8b1ff1dd.slice/crio-4a26890590d3a45554a97bdcd4bb031e31c7b5affb33bac2672d99c879ba497b WatchSource:0}: Error finding container 4a26890590d3a45554a97bdcd4bb031e31c7b5affb33bac2672d99c879ba497b: Status 404 returned error can't find the container with id 4a26890590d3a45554a97bdcd4bb031e31c7b5affb33bac2672d99c879ba497b Aug 05 18:13:43.904205 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.904165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293067792" event={"ID":"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd","Type":"ContainerStarted","Data":"4a26890590d3a45554a97bdcd4bb031e31c7b5affb33bac2672d99c879ba497b"} Aug 05 18:13:43.905347 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.905324 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:43.905472 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:43.905330 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18aaa1d618c49f952e963af1e8bd9237456ba1d9eca3739b8efe8fffbd4294a0" Aug 05 18:13:43.915835 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:13:43.915807 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-140246292930096\" is forbidden: User \"system:node:ip-10-0-143-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-58.ec2.internal' and this object" podUID="0ff6fd87-3e88-4053-bc59-5d809a167e77" pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:44.912803 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:44.912767 2575 generic.go:356] "Generic (PLEG): container finished" podID="9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" containerID="26d55433cbf7f02b2687b61f2e927cfde2849868309f6d953f17e56810a773d0" exitCode=0 Aug 05 18:13:44.913306 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:44.912861 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293067792" event={"ID":"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd","Type":"ContainerDied","Data":"26d55433cbf7f02b2687b61f2e927cfde2849868309f6d953f17e56810a773d0"} Aug 05 18:13:44.929228 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:13:44.929190 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-140246292930096\" is forbidden: User \"system:node:ip-10-0-143-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-58.ec2.internal' and this object" podUID="0ff6fd87-3e88-4053-bc59-5d809a167e77" pod="opendatahub/test-curl-7534-140246292930096" Aug 05 18:13:45.027341 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:45.027298 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ff6fd87-3e88-4053-bc59-5d809a167e77" path="/var/lib/kubelet/pods/0ff6fd87-3e88-4053-bc59-5d809a167e77/volumes" Aug 05 18:13:46.060066 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.060028 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.083133 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.083100 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7534-140246293067792_9c8cec83-24ed-46c0-8815-00ad8b1ff1dd/test-curl-7534-140246293067792/0.log" Aug 05 18:13:46.109824 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.109794 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:46.115180 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.115154 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:46.116925 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.116904 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrrsp\" (UniqueName: \"kubernetes.io/projected/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd-kube-api-access-qrrsp\") pod \"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd\" (UID: \"9c8cec83-24ed-46c0-8815-00ad8b1ff1dd\") " Aug 05 18:13:46.119040 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.119016 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd-kube-api-access-qrrsp" (OuterVolumeSpecName: "kube-api-access-qrrsp") pod "9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" (UID: "9c8cec83-24ed-46c0-8815-00ad8b1ff1dd"). InnerVolumeSpecName "kube-api-access-qrrsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:13:46.218513 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.218436 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrrsp\" (UniqueName: \"kubernetes.io/projected/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd-kube-api-access-qrrsp\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:13:46.237251 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.237217 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:46.237667 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.237653 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" containerName="test-curl-7534-140246293067792" Aug 05 18:13:46.237711 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.237669 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" containerName="test-curl-7534-140246293067792" Aug 05 18:13:46.237765 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.237753 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" containerName="test-curl-7534-140246293067792" Aug 05 18:13:46.240868 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.240852 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.246219 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.246188 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:46.319859 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.319826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn86q\" (UniqueName: \"kubernetes.io/projected/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f-kube-api-access-mn86q\") pod \"test-curl-7534-140246293067792\" (UID: \"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f\") " pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.421007 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.420965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn86q\" (UniqueName: \"kubernetes.io/projected/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f-kube-api-access-mn86q\") pod \"test-curl-7534-140246293067792\" (UID: \"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f\") " pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.429673 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.429633 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn86q\" (UniqueName: \"kubernetes.io/projected/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f-kube-api-access-mn86q\") pod \"test-curl-7534-140246293067792\" (UID: \"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f\") " pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.550989 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.550941 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.687760 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.687736 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:46.689471 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:13:46.689436 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f5eb476_8aa4_42d4_94ec_9a1e1ff8de5f.slice/crio-d19d2c3784ff34996ff559da4b884a48a0eaf69779a9747f0c1fbd1b8561faf2 WatchSource:0}: Error finding container d19d2c3784ff34996ff559da4b884a48a0eaf69779a9747f0c1fbd1b8561faf2: Status 404 returned error can't find the container with id d19d2c3784ff34996ff559da4b884a48a0eaf69779a9747f0c1fbd1b8561faf2 Aug 05 18:13:46.925665 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.925573 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:46.925665 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.925581 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a26890590d3a45554a97bdcd4bb031e31c7b5affb33bac2672d99c879ba497b" Aug 05 18:13:46.926782 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.926754 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293067792" event={"ID":"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f","Type":"ContainerStarted","Data":"d19d2c3784ff34996ff559da4b884a48a0eaf69779a9747f0c1fbd1b8561faf2"} Aug 05 18:13:46.937908 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:46.937879 2575 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7534-140246293067792" oldPodUID="9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" Aug 05 18:13:47.028182 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:47.028157 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c8cec83-24ed-46c0-8815-00ad8b1ff1dd" path="/var/lib/kubelet/pods/9c8cec83-24ed-46c0-8815-00ad8b1ff1dd/volumes" Aug 05 18:13:47.940550 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:47.940514 2575 generic.go:356] "Generic (PLEG): container finished" podID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" containerID="bd266412f72c6dd39dab44ee5e3b14a1d647bc0b34c07691002188762ed18512" exitCode=0 Aug 05 18:13:47.941037 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:47.940604 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293067792" event={"ID":"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f","Type":"ContainerDied","Data":"bd266412f72c6dd39dab44ee5e3b14a1d647bc0b34c07691002188762ed18512"} Aug 05 18:13:49.081770 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.081738 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:49.101387 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.101357 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7534-140246293067792_2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f/test-curl-7534-140246293067792/0.log" Aug 05 18:13:49.126173 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.126141 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:49.128359 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.128338 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-140246293067792"] Aug 05 18:13:49.149994 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.149969 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn86q\" (UniqueName: \"kubernetes.io/projected/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f-kube-api-access-mn86q\") pod \"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f\" (UID: \"2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f\") " Aug 05 18:13:49.152068 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.152026 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f-kube-api-access-mn86q" (OuterVolumeSpecName: "kube-api-access-mn86q") pod "2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" (UID: "2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f"). InnerVolumeSpecName "kube-api-access-mn86q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:13:49.250418 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.250330 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:49.250743 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.250729 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" containerName="test-curl-7534-140246293067792" Aug 05 18:13:49.250788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.250750 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" containerName="test-curl-7534-140246293067792" Aug 05 18:13:49.250837 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.250731 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn86q\" (UniqueName: \"kubernetes.io/projected/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f-kube-api-access-mn86q\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:13:49.250879 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.250825 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" containerName="test-curl-7534-140246293067792" Aug 05 18:13:49.253859 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.253844 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:49.260432 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.260402 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:49.351840 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.351802 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgczl\" (UniqueName: \"kubernetes.io/projected/18260c07-9697-4c8c-afc5-14ec186918a9-kube-api-access-sgczl\") pod \"test-curl-7534-140246293068800\" (UID: \"18260c07-9697-4c8c-afc5-14ec186918a9\") " pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:49.453376 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.453334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgczl\" (UniqueName: \"kubernetes.io/projected/18260c07-9697-4c8c-afc5-14ec186918a9-kube-api-access-sgczl\") pod \"test-curl-7534-140246293068800\" (UID: \"18260c07-9697-4c8c-afc5-14ec186918a9\") " pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:49.466853 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.466832 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgczl\" (UniqueName: \"kubernetes.io/projected/18260c07-9697-4c8c-afc5-14ec186918a9-kube-api-access-sgczl\") pod \"test-curl-7534-140246293068800\" (UID: \"18260c07-9697-4c8c-afc5-14ec186918a9\") " pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:49.564468 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.564420 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:49.711501 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.711474 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:49.713321 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:13:49.713291 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18260c07_9697_4c8c_afc5_14ec186918a9.slice/crio-9b04064c41ef09e88f6207c32353dcff1799223335c97d9f019a2280729c4e48 WatchSource:0}: Error finding container 9b04064c41ef09e88f6207c32353dcff1799223335c97d9f019a2280729c4e48: Status 404 returned error can't find the container with id 9b04064c41ef09e88f6207c32353dcff1799223335c97d9f019a2280729c4e48 Aug 05 18:13:49.953322 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.953241 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:49.953322 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.953260 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d19d2c3784ff34996ff559da4b884a48a0eaf69779a9747f0c1fbd1b8561faf2" Aug 05 18:13:49.954514 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:49.954487 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293068800" event={"ID":"18260c07-9697-4c8c-afc5-14ec186918a9","Type":"ContainerStarted","Data":"9b04064c41ef09e88f6207c32353dcff1799223335c97d9f019a2280729c4e48"} Aug 05 18:13:49.963679 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:13:49.963643 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-140246293067792\" is forbidden: User \"system:node:ip-10-0-143-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-58.ec2.internal' and this object" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:50.961217 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:50.961183 2575 generic.go:356] "Generic (PLEG): container finished" podID="18260c07-9697-4c8c-afc5-14ec186918a9" containerID="1322bae1a16d86bb03046b485e07e29286f9268ac09c4935ca99fb3bf7b6dff5" exitCode=0 Aug 05 18:13:50.961652 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:50.961267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293068800" event={"ID":"18260c07-9697-4c8c-afc5-14ec186918a9","Type":"ContainerDied","Data":"1322bae1a16d86bb03046b485e07e29286f9268ac09c4935ca99fb3bf7b6dff5"} Aug 05 18:13:50.963074 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:13:50.963029 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-140246293067792\" is forbidden: User \"system:node:ip-10-0-143-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-58.ec2.internal' and this object" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" pod="opendatahub/test-curl-7534-140246293067792" Aug 05 18:13:51.026788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:51.026756 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f" path="/var/lib/kubelet/pods/2f5eb476-8aa4-42d4-94ec-9a1e1ff8de5f/volumes" Aug 05 18:13:52.103252 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.103225 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.122829 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.122797 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7534-140246293068800_18260c07-9697-4c8c-afc5-14ec186918a9/test-curl-7534-140246293068800/0.log" Aug 05 18:13:52.144933 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.144891 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:52.148416 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.148388 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:52.271116 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.271023 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:52.271436 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.271423 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18260c07-9697-4c8c-afc5-14ec186918a9" containerName="test-curl-7534-140246293068800" Aug 05 18:13:52.271476 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.271438 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="18260c07-9697-4c8c-afc5-14ec186918a9" containerName="test-curl-7534-140246293068800" Aug 05 18:13:52.271529 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.271519 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="18260c07-9697-4c8c-afc5-14ec186918a9" containerName="test-curl-7534-140246293068800" Aug 05 18:13:52.274534 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.274517 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.279309 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.279286 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:52.279501 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.279480 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgczl\" (UniqueName: \"kubernetes.io/projected/18260c07-9697-4c8c-afc5-14ec186918a9-kube-api-access-sgczl\") pod \"18260c07-9697-4c8c-afc5-14ec186918a9\" (UID: \"18260c07-9697-4c8c-afc5-14ec186918a9\") " Aug 05 18:13:52.281611 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.281583 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18260c07-9697-4c8c-afc5-14ec186918a9-kube-api-access-sgczl" (OuterVolumeSpecName: "kube-api-access-sgczl") pod "18260c07-9697-4c8c-afc5-14ec186918a9" (UID: "18260c07-9697-4c8c-afc5-14ec186918a9"). InnerVolumeSpecName "kube-api-access-sgczl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:13:52.380368 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.380333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkhg\" (UniqueName: \"kubernetes.io/projected/6080c488-feea-4a2d-926a-641fa7f226f6-kube-api-access-7dkhg\") pod \"test-curl-7534-140246293068800\" (UID: \"6080c488-feea-4a2d-926a-641fa7f226f6\") " pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.380532 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.380437 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgczl\" (UniqueName: \"kubernetes.io/projected/18260c07-9697-4c8c-afc5-14ec186918a9-kube-api-access-sgczl\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:13:52.481232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.481184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkhg\" (UniqueName: \"kubernetes.io/projected/6080c488-feea-4a2d-926a-641fa7f226f6-kube-api-access-7dkhg\") pod \"test-curl-7534-140246293068800\" (UID: \"6080c488-feea-4a2d-926a-641fa7f226f6\") " pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.489397 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.489362 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkhg\" (UniqueName: \"kubernetes.io/projected/6080c488-feea-4a2d-926a-641fa7f226f6-kube-api-access-7dkhg\") pod \"test-curl-7534-140246293068800\" (UID: \"6080c488-feea-4a2d-926a-641fa7f226f6\") " pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.597322 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.597281 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.727529 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.727498 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:52.729154 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:13:52.729116 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6080c488_feea_4a2d_926a_641fa7f226f6.slice/crio-9133a6453457382b31c5fd0d0da701ff524e2d5dc9ae11c0989cc0c5b27cc4de WatchSource:0}: Error finding container 9133a6453457382b31c5fd0d0da701ff524e2d5dc9ae11c0989cc0c5b27cc4de: Status 404 returned error can't find the container with id 9133a6453457382b31c5fd0d0da701ff524e2d5dc9ae11c0989cc0c5b27cc4de Aug 05 18:13:52.980401 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.980310 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:52.980401 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.980339 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b04064c41ef09e88f6207c32353dcff1799223335c97d9f019a2280729c4e48" Aug 05 18:13:52.981576 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.981547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293068800" event={"ID":"6080c488-feea-4a2d-926a-641fa7f226f6","Type":"ContainerStarted","Data":"9133a6453457382b31c5fd0d0da701ff524e2d5dc9ae11c0989cc0c5b27cc4de"} Aug 05 18:13:52.991565 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:52.991543 2575 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7534-140246293068800" oldPodUID="18260c07-9697-4c8c-afc5-14ec186918a9" podUID="6080c488-feea-4a2d-926a-641fa7f226f6" Aug 05 18:13:53.026761 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:53.026725 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18260c07-9697-4c8c-afc5-14ec186918a9" path="/var/lib/kubelet/pods/18260c07-9697-4c8c-afc5-14ec186918a9/volumes" Aug 05 18:13:53.988162 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:53.988122 2575 generic.go:356] "Generic (PLEG): container finished" podID="6080c488-feea-4a2d-926a-641fa7f226f6" containerID="61fbb41e752844a99bc3e5a4520bb27c9aa8d24013acace271bdd0c5de56538a" exitCode=0 Aug 05 18:13:53.988540 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:53.988208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7534-140246293068800" event={"ID":"6080c488-feea-4a2d-926a-641fa7f226f6","Type":"ContainerDied","Data":"61fbb41e752844a99bc3e5a4520bb27c9aa8d24013acace271bdd0c5de56538a"} Aug 05 18:13:55.132649 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.132623 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:55.151453 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.151422 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7534-140246293068800_6080c488-feea-4a2d-926a-641fa7f226f6/test-curl-7534-140246293068800/0.log" Aug 05 18:13:55.172695 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.172666 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:55.176464 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.176439 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7534-140246293068800"] Aug 05 18:13:55.305596 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.305566 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dkhg\" (UniqueName: \"kubernetes.io/projected/6080c488-feea-4a2d-926a-641fa7f226f6-kube-api-access-7dkhg\") pod \"6080c488-feea-4a2d-926a-641fa7f226f6\" (UID: \"6080c488-feea-4a2d-926a-641fa7f226f6\") " Aug 05 18:13:55.307767 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.307739 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6080c488-feea-4a2d-926a-641fa7f226f6-kube-api-access-7dkhg" (OuterVolumeSpecName: "kube-api-access-7dkhg") pod "6080c488-feea-4a2d-926a-641fa7f226f6" (UID: "6080c488-feea-4a2d-926a-641fa7f226f6"). InnerVolumeSpecName "kube-api-access-7dkhg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:13:55.406726 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:55.406691 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dkhg\" (UniqueName: \"kubernetes.io/projected/6080c488-feea-4a2d-926a-641fa7f226f6-kube-api-access-7dkhg\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:13:56.002128 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:56.002095 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:56.002128 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:56.002108 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9133a6453457382b31c5fd0d0da701ff524e2d5dc9ae11c0989cc0c5b27cc4de" Aug 05 18:13:56.012252 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:13:56.012210 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7534-140246293068800\" is forbidden: User \"system:node:ip-10-0-143-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-58.ec2.internal' and this object" podUID="6080c488-feea-4a2d-926a-641fa7f226f6" pod="opendatahub/test-curl-7534-140246293068800" Aug 05 18:13:57.026477 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:13:57.026446 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6080c488-feea-4a2d-926a-641fa7f226f6" path="/var/lib/kubelet/pods/6080c488-feea-4a2d-926a-641fa7f226f6/volumes" Aug 05 18:14:00.676120 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.676038 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv"] Aug 05 18:14:00.676490 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.676473 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6080c488-feea-4a2d-926a-641fa7f226f6" containerName="test-curl-7534-140246293068800" Aug 05 18:14:00.676490 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.676485 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6080c488-feea-4a2d-926a-641fa7f226f6" containerName="test-curl-7534-140246293068800" Aug 05 18:14:00.676569 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.676563 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6080c488-feea-4a2d-926a-641fa7f226f6" containerName="test-curl-7534-140246293068800" Aug 05 18:14:00.679833 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.679809 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.681930 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.681901 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 18:14:00.682170 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.682153 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 05 18:14:00.682525 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.682509 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-4babb1-serving-cert\"" Aug 05 18:14:00.682525 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.682520 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ql6kj\"" Aug 05 18:14:00.690521 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.690497 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv"] Aug 05 18:14:00.859370 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.859329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5pwl\" (UniqueName: \"kubernetes.io/projected/0407a903-8ed9-47ed-bdcf-fb91dec3620e-kube-api-access-j5pwl\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.859568 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.859399 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0407a903-8ed9-47ed-bdcf-fb91dec3620e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.859568 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.859501 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0407a903-8ed9-47ed-bdcf-fb91dec3620e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.960616 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.960526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0407a903-8ed9-47ed-bdcf-fb91dec3620e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.960616 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.960589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5pwl\" (UniqueName: \"kubernetes.io/projected/0407a903-8ed9-47ed-bdcf-fb91dec3620e-kube-api-access-j5pwl\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.960808 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.960623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0407a903-8ed9-47ed-bdcf-fb91dec3620e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.961276 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.961255 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0407a903-8ed9-47ed-bdcf-fb91dec3620e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.963121 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.963100 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0407a903-8ed9-47ed-bdcf-fb91dec3620e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.968214 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.968188 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5pwl\" (UniqueName: \"kubernetes.io/projected/0407a903-8ed9-47ed-bdcf-fb91dec3620e-kube-api-access-j5pwl\") pod \"maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:00.993666 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:00.993636 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:01.135340 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:01.135298 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv"] Aug 05 18:14:01.135799 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:14:01.135766 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0407a903_8ed9_47ed_bdcf_fb91dec3620e.slice/crio-2f98af969190475725bb49627f1c59a9e2f3f03264717024a70fe3c1da7a78ef WatchSource:0}: Error finding container 2f98af969190475725bb49627f1c59a9e2f3f03264717024a70fe3c1da7a78ef: Status 404 returned error can't find the container with id 2f98af969190475725bb49627f1c59a9e2f3f03264717024a70fe3c1da7a78ef Aug 05 18:14:02.039864 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:02.039762 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" event={"ID":"0407a903-8ed9-47ed-bdcf-fb91dec3620e","Type":"ContainerStarted","Data":"f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f"} Aug 05 18:14:02.040293 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:02.039910 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:02.040293 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:02.039952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" event={"ID":"0407a903-8ed9-47ed-bdcf-fb91dec3620e","Type":"ContainerStarted","Data":"2f98af969190475725bb49627f1c59a9e2f3f03264717024a70fe3c1da7a78ef"} Aug 05 18:14:02.057001 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:02.056947 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" podStartSLOduration=1.5869427900000002 podStartE2EDuration="2.056932085s" totalImagesPullingTime="469.989295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:02.054760786 +0000 UTC m=+3279.455989625" watchObservedRunningTime="2026-08-05 18:14:02.056932085 +0000 UTC m=+3279.458160899" Aug 05 18:14:03.766534 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:03.766493 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:14:08.050897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:08.050865 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:14:23.368667 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:23.368633 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:14:23.374367 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:23.374343 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:14:23.388637 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:23.388607 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:14:23.394544 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:23.394512 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:14:29.662647 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.662613 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt"] Aug 05 18:14:29.668013 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.667989 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.670138 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.670112 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a83758-serving-cert\"" Aug 05 18:14:29.676214 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.676180 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt"] Aug 05 18:14:29.728453 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.728420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tfkm\" (UniqueName: \"kubernetes.io/projected/31fad774-c141-4730-9d58-786652849952-kube-api-access-6tfkm\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.728643 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.728460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fad774-c141-4730-9d58-786652849952-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.728643 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.728493 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fad774-c141-4730-9d58-786652849952-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.829605 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.829571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tfkm\" (UniqueName: \"kubernetes.io/projected/31fad774-c141-4730-9d58-786652849952-kube-api-access-6tfkm\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.829816 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.829610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fad774-c141-4730-9d58-786652849952-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.829816 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.829649 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fad774-c141-4730-9d58-786652849952-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.830425 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.830399 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fad774-c141-4730-9d58-786652849952-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.832100 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.832082 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fad774-c141-4730-9d58-786652849952-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.837528 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.837506 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tfkm\" (UniqueName: \"kubernetes.io/projected/31fad774-c141-4730-9d58-786652849952-kube-api-access-6tfkm\") pod \"maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:29.981882 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:29.981796 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:30.352244 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:30.352215 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt"] Aug 05 18:14:30.354263 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:14:30.354233 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fad774_c141_4730_9d58_786652849952.slice/crio-abb154ae3df21c28018d285b80b76c5f7066f8dbf35237b48660024f73b136e1 WatchSource:0}: Error finding container abb154ae3df21c28018d285b80b76c5f7066f8dbf35237b48660024f73b136e1: Status 404 returned error can't find the container with id abb154ae3df21c28018d285b80b76c5f7066f8dbf35237b48660024f73b136e1 Aug 05 18:14:31.217541 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:31.217425 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" event={"ID":"31fad774-c141-4730-9d58-786652849952","Type":"ContainerStarted","Data":"4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e"} Aug 05 18:14:31.217954 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:31.217610 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:14:31.217954 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:31.217650 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" event={"ID":"31fad774-c141-4730-9d58-786652849952","Type":"ContainerStarted","Data":"abb154ae3df21c28018d285b80b76c5f7066f8dbf35237b48660024f73b136e1"} Aug 05 18:14:31.236642 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:31.236587 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" podStartSLOduration=1.791834261 podStartE2EDuration="2.236564902s" totalImagesPullingTime="444.730641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:14:31.233709267 +0000 UTC m=+3308.634938083" watchObservedRunningTime="2026-08-05 18:14:31.236564902 +0000 UTC m=+3308.637793717" Aug 05 18:14:33.170048 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:33.170014 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:14:37.228720 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:14:37.228688 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:15:00.141110 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.141028 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7"] Aug 05 18:15:00.145434 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.145406 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" Aug 05 18:15:00.145602 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.145559 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm"] Aug 05 18:15:00.147510 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.147488 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:15:00.149164 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.149142 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s"] Aug 05 18:15:00.149287 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.149223 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" Aug 05 18:15:00.152475 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.152454 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx"] Aug 05 18:15:00.153606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.153041 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" Aug 05 18:15:00.157607 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.156885 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l"] Aug 05 18:15:00.158801 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.158754 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" Aug 05 18:15:00.159525 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.159507 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" Aug 05 18:15:00.160976 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.160958 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7"] Aug 05 18:15:00.161156 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.161141 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s"] Aug 05 18:15:00.161810 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.161321 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" Aug 05 18:15:00.164283 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.164258 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm"] Aug 05 18:15:00.172311 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.169107 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx"] Aug 05 18:15:00.172311 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.171571 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" Aug 05 18:15:00.172311 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.172115 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l"] Aug 05 18:15:00.214438 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.214230 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" Aug 05 18:15:00.226524 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.226383 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" Aug 05 18:15:00.269222 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.269120 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" Aug 05 18:15:00.359510 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.359444 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm"] Aug 05 18:15:00.386843 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.386813 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7"] Aug 05 18:15:00.389873 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:15:00.389834 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353aaaab_f9ce_4cdf_ae11_915f6ca04861.slice/crio-9414d5be95104d0957fd2ceba399d4737b6df71234887a7893daaa238921e546 WatchSource:0}: Error finding container 9414d5be95104d0957fd2ceba399d4737b6df71234887a7893daaa238921e546: Status 404 returned error can't find the container with id 9414d5be95104d0957fd2ceba399d4737b6df71234887a7893daaa238921e546 Aug 05 18:15:00.404446 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.404399 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" event={"ID":"176f3f63-4035-4359-ad52-8202be20c561","Type":"ContainerStarted","Data":"9264ddc9931eeed56fadb52ad0b4aa9ea677465dc31f9fdf592a8fce1cdbd9f4"} Aug 05 18:15:00.432684 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.432611 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s"] Aug 05 18:15:00.437773 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:15:00.437745 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6246a525_8ae0_43b0_864a_9512c505fd94.slice/crio-6d9b6e9c6d9d2c3c33bc447b22233a66715f62c0981f92e5d4b4c76e0dc49b68 WatchSource:0}: Error finding container 6d9b6e9c6d9d2c3c33bc447b22233a66715f62c0981f92e5d4b4c76e0dc49b68: Status 404 returned error can't find the container with id 6d9b6e9c6d9d2c3c33bc447b22233a66715f62c0981f92e5d4b4c76e0dc49b68 Aug 05 18:15:00.458688 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.458642 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx"] Aug 05 18:15:00.462665 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:15:00.462630 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f8f6fd_0682_413c_8e7c_a8f445526a8d.slice/crio-8ab8abab0b278e1079ed4b7328a98ac49259731b2de8788a1673762be5e41422 WatchSource:0}: Error finding container 8ab8abab0b278e1079ed4b7328a98ac49259731b2de8788a1673762be5e41422: Status 404 returned error can't find the container with id 8ab8abab0b278e1079ed4b7328a98ac49259731b2de8788a1673762be5e41422 Aug 05 18:15:00.686804 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:00.686780 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l"] Aug 05 18:15:00.688592 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:15:00.688568 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f82df0d_27cf_4f94_8d10_16ada09b3376.slice/crio-4a8ece2bf27a01f9252259874441174fc209c8f57fb84b5a496f0afb5738edbd WatchSource:0}: Error finding container 4a8ece2bf27a01f9252259874441174fc209c8f57fb84b5a496f0afb5738edbd: Status 404 returned error can't find the container with id 4a8ece2bf27a01f9252259874441174fc209c8f57fb84b5a496f0afb5738edbd Aug 05 18:15:01.412184 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.412157 2575 generic.go:356] "Generic (PLEG): container finished" podID="176f3f63-4035-4359-ad52-8202be20c561" containerID="942b926a846a946ab3eeedf29c56965dac47ee8c72e336914e0e551b80ea3890" exitCode=0 Aug 05 18:15:01.412529 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.412222 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" event={"ID":"176f3f63-4035-4359-ad52-8202be20c561","Type":"ContainerDied","Data":"942b926a846a946ab3eeedf29c56965dac47ee8c72e336914e0e551b80ea3890"} Aug 05 18:15:01.413719 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.413694 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" event={"ID":"353aaaab-f9ce-4cdf-ae11-915f6ca04861","Type":"ContainerStarted","Data":"29dcaa6d0f552f7d50bcd72968185c0a5b6cac8b90ee404339a9fc6b77ab55e9"} Aug 05 18:15:01.413851 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.413739 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" event={"ID":"353aaaab-f9ce-4cdf-ae11-915f6ca04861","Type":"ContainerStarted","Data":"9414d5be95104d0957fd2ceba399d4737b6df71234887a7893daaa238921e546"} Aug 05 18:15:01.415232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.415206 2575 generic.go:356] "Generic (PLEG): container finished" podID="02f8f6fd-0682-413c-8e7c-a8f445526a8d" containerID="732c5cc99f3431fbe499aa1e20e531173e03f88415d484e748e257668ae168b4" exitCode=0 Aug 05 18:15:01.415325 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.415293 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" event={"ID":"02f8f6fd-0682-413c-8e7c-a8f445526a8d","Type":"ContainerDied","Data":"732c5cc99f3431fbe499aa1e20e531173e03f88415d484e748e257668ae168b4"} Aug 05 18:15:01.415389 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.415346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" event={"ID":"02f8f6fd-0682-413c-8e7c-a8f445526a8d","Type":"ContainerStarted","Data":"8ab8abab0b278e1079ed4b7328a98ac49259731b2de8788a1673762be5e41422"} Aug 05 18:15:01.416758 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.416737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" event={"ID":"6f82df0d-27cf-4f94-8d10-16ada09b3376","Type":"ContainerStarted","Data":"1544895094be94d4e9eac437e87837aa14b32f97f6fcd04ddd28d0613f3458b3"} Aug 05 18:15:01.416866 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.416773 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" event={"ID":"6f82df0d-27cf-4f94-8d10-16ada09b3376","Type":"ContainerStarted","Data":"4a8ece2bf27a01f9252259874441174fc209c8f57fb84b5a496f0afb5738edbd"} Aug 05 18:15:01.418152 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.418131 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" event={"ID":"6246a525-8ae0-43b0-864a-9512c505fd94","Type":"ContainerStarted","Data":"a3112a9d1686d4bd51d489a2bcf591a334706ea94cd7c5fd560c7b7f5c427e4c"} Aug 05 18:15:01.418247 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.418168 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" event={"ID":"6246a525-8ae0-43b0-864a-9512c505fd94","Type":"ContainerStarted","Data":"6d9b6e9c6d9d2c3c33bc447b22233a66715f62c0981f92e5d4b4c76e0dc49b68"} Aug 05 18:15:01.442638 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.442574 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" podStartSLOduration=1.442559858 podStartE2EDuration="1.442559858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:01.440313333 +0000 UTC m=+3338.841542149" watchObservedRunningTime="2026-08-05 18:15:01.442559858 +0000 UTC m=+3338.843788651" Aug 05 18:15:01.467005 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.466947 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" podStartSLOduration=1.466934444 podStartE2EDuration="1.466934444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:01.465242115 +0000 UTC m=+3338.866470929" watchObservedRunningTime="2026-08-05 18:15:01.466934444 +0000 UTC m=+3338.868163258" Aug 05 18:15:01.478763 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:01.478722 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" podStartSLOduration=1.478709372 podStartE2EDuration="1.478709372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:01.477612598 +0000 UTC m=+3338.878841413" watchObservedRunningTime="2026-08-05 18:15:01.478709372 +0000 UTC m=+3338.879938187" Aug 05 18:15:02.425663 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.425630 2575 generic.go:356] "Generic (PLEG): container finished" podID="6f82df0d-27cf-4f94-8d10-16ada09b3376" containerID="1544895094be94d4e9eac437e87837aa14b32f97f6fcd04ddd28d0613f3458b3" exitCode=0 Aug 05 18:15:02.426155 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.425720 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" event={"ID":"6f82df0d-27cf-4f94-8d10-16ada09b3376","Type":"ContainerDied","Data":"1544895094be94d4e9eac437e87837aa14b32f97f6fcd04ddd28d0613f3458b3"} Aug 05 18:15:02.427286 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.427261 2575 generic.go:356] "Generic (PLEG): container finished" podID="6246a525-8ae0-43b0-864a-9512c505fd94" containerID="a3112a9d1686d4bd51d489a2bcf591a334706ea94cd7c5fd560c7b7f5c427e4c" exitCode=0 Aug 05 18:15:02.427392 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.427327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" event={"ID":"6246a525-8ae0-43b0-864a-9512c505fd94","Type":"ContainerDied","Data":"a3112a9d1686d4bd51d489a2bcf591a334706ea94cd7c5fd560c7b7f5c427e4c"} Aug 05 18:15:02.428788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.428765 2575 generic.go:356] "Generic (PLEG): container finished" podID="353aaaab-f9ce-4cdf-ae11-915f6ca04861" containerID="29dcaa6d0f552f7d50bcd72968185c0a5b6cac8b90ee404339a9fc6b77ab55e9" exitCode=0 Aug 05 18:15:02.428890 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.428842 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" event={"ID":"353aaaab-f9ce-4cdf-ae11-915f6ca04861","Type":"ContainerDied","Data":"29dcaa6d0f552f7d50bcd72968185c0a5b6cac8b90ee404339a9fc6b77ab55e9"} Aug 05 18:15:02.566522 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.566495 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" Aug 05 18:15:02.603386 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:02.603356 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" Aug 05 18:15:03.436003 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.435976 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" Aug 05 18:15:03.436003 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.435989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm" event={"ID":"176f3f63-4035-4359-ad52-8202be20c561","Type":"ContainerDied","Data":"9264ddc9931eeed56fadb52ad0b4aa9ea677465dc31f9fdf592a8fce1cdbd9f4"} Aug 05 18:15:03.436532 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.436041 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9264ddc9931eeed56fadb52ad0b4aa9ea677465dc31f9fdf592a8fce1cdbd9f4" Aug 05 18:15:03.437462 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.437438 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" event={"ID":"02f8f6fd-0682-413c-8e7c-a8f445526a8d","Type":"ContainerDied","Data":"8ab8abab0b278e1079ed4b7328a98ac49259731b2de8788a1673762be5e41422"} Aug 05 18:15:03.437562 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.437492 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ab8abab0b278e1079ed4b7328a98ac49259731b2de8788a1673762be5e41422" Aug 05 18:15:03.437562 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.437448 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29765895-p2psx" Aug 05 18:15:03.578232 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.578207 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" Aug 05 18:15:03.640074 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.629215 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" Aug 05 18:15:03.669064 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:03.669028 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" Aug 05 18:15:04.445624 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.445589 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" Aug 05 18:15:04.445624 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.445614 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l" event={"ID":"6f82df0d-27cf-4f94-8d10-16ada09b3376","Type":"ContainerDied","Data":"4a8ece2bf27a01f9252259874441174fc209c8f57fb84b5a496f0afb5738edbd"} Aug 05 18:15:04.446176 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.445660 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a8ece2bf27a01f9252259874441174fc209c8f57fb84b5a496f0afb5738edbd" Aug 05 18:15:04.447251 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.447230 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" Aug 05 18:15:04.447375 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.447277 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s" event={"ID":"6246a525-8ae0-43b0-864a-9512c505fd94","Type":"ContainerDied","Data":"6d9b6e9c6d9d2c3c33bc447b22233a66715f62c0981f92e5d4b4c76e0dc49b68"} Aug 05 18:15:04.447375 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.447334 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d9b6e9c6d9d2c3c33bc447b22233a66715f62c0981f92e5d4b4c76e0dc49b68" Aug 05 18:15:04.448684 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.448664 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" event={"ID":"353aaaab-f9ce-4cdf-ae11-915f6ca04861","Type":"ContainerDied","Data":"9414d5be95104d0957fd2ceba399d4737b6df71234887a7893daaa238921e546"} Aug 05 18:15:04.448793 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.448693 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7" Aug 05 18:15:04.448793 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:04.448698 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9414d5be95104d0957fd2ceba399d4737b6df71234887a7893daaa238921e546" Aug 05 18:15:12.668426 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668381 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq"] Aug 05 18:15:12.668855 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668842 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="176f3f63-4035-4359-ad52-8202be20c561" containerName="cleanup" Aug 05 18:15:12.668896 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668856 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="176f3f63-4035-4359-ad52-8202be20c561" containerName="cleanup" Aug 05 18:15:12.668896 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668864 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f82df0d-27cf-4f94-8d10-16ada09b3376" containerName="cleanup" Aug 05 18:15:12.668896 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668870 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f82df0d-27cf-4f94-8d10-16ada09b3376" containerName="cleanup" Aug 05 18:15:12.668896 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668895 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f8f6fd-0682-413c-8e7c-a8f445526a8d" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668901 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f8f6fd-0682-413c-8e7c-a8f445526a8d" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668909 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6246a525-8ae0-43b0-864a-9512c505fd94" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668915 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6246a525-8ae0-43b0-864a-9512c505fd94" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668927 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353aaaab-f9ce-4cdf-ae11-915f6ca04861" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.668932 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="353aaaab-f9ce-4cdf-ae11-915f6ca04861" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.669003 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f82df0d-27cf-4f94-8d10-16ada09b3376" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.669011 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="353aaaab-f9ce-4cdf-ae11-915f6ca04861" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.669021 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6246a525-8ae0-43b0-864a-9512c505fd94" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.669028 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="176f3f63-4035-4359-ad52-8202be20c561" containerName="cleanup" Aug 05 18:15:12.669043 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.669033 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f8f6fd-0682-413c-8e7c-a8f445526a8d" containerName="cleanup" Aug 05 18:15:12.672410 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.672391 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.674542 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.674515 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4babb1\"/\"openshift-service-ca.crt\"" Aug 05 18:15:12.674712 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.674583 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4babb1\"/\"default-dockercfg-z5v5z\"" Aug 05 18:15:12.675474 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.675451 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4babb1\"/\"ipp-route-4babb1-kserve-self-signed-certs\"" Aug 05 18:15:12.675897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.675877 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4babb1\"/\"kube-root-ca.crt\"" Aug 05 18:15:12.678989 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.678963 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq"] Aug 05 18:15:12.719256 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.719226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-dshm\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.719256 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.719256 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-model-cache\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.719481 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.719276 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-home\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.719481 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.719298 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pxxc\" (UniqueName: \"kubernetes.io/projected/5967882b-ff2a-4e0c-9baa-2bbd72a87633-kube-api-access-6pxxc\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.719481 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.719389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tmp-dir\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.719481 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.719422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tls-certs\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.820866 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.820825 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-dshm\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.820866 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.820865 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-model-cache\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821110 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.820886 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-home\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821160 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.821102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pxxc\" (UniqueName: \"kubernetes.io/projected/5967882b-ff2a-4e0c-9baa-2bbd72a87633-kube-api-access-6pxxc\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821210 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.821155 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tmp-dir\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821210 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.821194 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tls-certs\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821361 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.821327 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-model-cache\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821361 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.821347 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-home\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.821565 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.821544 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tmp-dir\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.823103 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.823077 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-dshm\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.823562 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.823546 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tls-certs\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.828408 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.828387 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pxxc\" (UniqueName: \"kubernetes.io/projected/5967882b-ff2a-4e0c-9baa-2bbd72a87633-kube-api-access-6pxxc\") pod \"ipp-route-4babb1-kserve-5d557b4cd7-xzrnq\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:12.985801 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:12.985702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:13.128758 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:13.128730 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq"] Aug 05 18:15:13.129610 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:15:13.129584 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5967882b_ff2a_4e0c_9baa_2bbd72a87633.slice/crio-a9d0554a1a44a46b5dce851612c44a722a63b24ed5e6a332c24aa3b4ab3a17d7 WatchSource:0}: Error finding container a9d0554a1a44a46b5dce851612c44a722a63b24ed5e6a332c24aa3b4ab3a17d7: Status 404 returned error can't find the container with id a9d0554a1a44a46b5dce851612c44a722a63b24ed5e6a332c24aa3b4ab3a17d7 Aug 05 18:15:13.514343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:13.514309 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" event={"ID":"5967882b-ff2a-4e0c-9baa-2bbd72a87633","Type":"ContainerStarted","Data":"1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af"} Aug 05 18:15:13.514343 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:13.514353 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" event={"ID":"5967882b-ff2a-4e0c-9baa-2bbd72a87633","Type":"ContainerStarted","Data":"a9d0554a1a44a46b5dce851612c44a722a63b24ed5e6a332c24aa3b4ab3a17d7"} Aug 05 18:15:13.532591 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:13.532541 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" podStartSLOduration=1.532527044 podStartE2EDuration="1.532527044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:13.530148834 +0000 UTC m=+3350.931377668" watchObservedRunningTime="2026-08-05 18:15:13.532527044 +0000 UTC m=+3350.933755859" Aug 05 18:15:15.373742 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:15.373701 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:15:22.985924 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:22.985889 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:22.986334 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:22.985936 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:22.998499 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:22.998474 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:23.586529 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:23.586498 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:15:39.362580 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:39.362547 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:15:55.750359 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.750322 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9"] Aug 05 18:15:55.754468 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.754442 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:55.756812 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.756785 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-fb84d0-dockercfg-rxzvd\"" Aug 05 18:15:55.756939 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.756890 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-fb84d0\"" Aug 05 18:15:55.764782 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.764753 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9"] Aug 05 18:15:55.850002 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.849956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd86d37b-133c-4145-8d82-446c89692a31-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:55.850189 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.850047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqzbg\" (UniqueName: \"kubernetes.io/projected/cd86d37b-133c-4145-8d82-446c89692a31-kube-api-access-xqzbg\") pod \"payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:55.951292 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.951252 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd86d37b-133c-4145-8d82-446c89692a31-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:55.951484 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.951344 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqzbg\" (UniqueName: \"kubernetes.io/projected/cd86d37b-133c-4145-8d82-446c89692a31-kube-api-access-xqzbg\") pod \"payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:55.952078 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.952027 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd86d37b-133c-4145-8d82-446c89692a31-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:55.959295 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:55.959272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqzbg\" (UniqueName: \"kubernetes.io/projected/cd86d37b-133c-4145-8d82-446c89692a31-kube-api-access-xqzbg\") pod \"payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:56.065610 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:56.065574 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:56.209731 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:56.209702 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9"] Aug 05 18:15:56.210942 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:15:56.210910 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd86d37b_133c_4145_8d82_446c89692a31.slice/crio-767bc5fe593aebf3d2526628af3ff1801f163a4042eacc6832a11a5760c96b1f WatchSource:0}: Error finding container 767bc5fe593aebf3d2526628af3ff1801f163a4042eacc6832a11a5760c96b1f: Status 404 returned error can't find the container with id 767bc5fe593aebf3d2526628af3ff1801f163a4042eacc6832a11a5760c96b1f Aug 05 18:15:56.784800 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:56.784753 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" event={"ID":"cd86d37b-133c-4145-8d82-446c89692a31","Type":"ContainerStarted","Data":"cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e"} Aug 05 18:15:56.785289 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:56.784842 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" event={"ID":"cd86d37b-133c-4145-8d82-446c89692a31","Type":"ContainerStarted","Data":"767bc5fe593aebf3d2526628af3ff1801f163a4042eacc6832a11a5760c96b1f"} Aug 05 18:15:56.785289 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:56.784865 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:15:56.799646 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:15:56.799581 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" podStartSLOduration=1.799565227 podStartE2EDuration="1.799565227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:56.79896472 +0000 UTC m=+3394.200193535" watchObservedRunningTime="2026-08-05 18:15:56.799565227 +0000 UTC m=+3394.200794041" Aug 05 18:16:00.870994 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:00.870952 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:16:07.793579 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:07.793548 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:16:20.358477 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.358393 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h"] Aug 05 18:16:20.362254 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.362236 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:20.364425 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.364404 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:16:20.369228 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.369198 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h"] Aug 05 18:16:20.393567 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.393530 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d7756f-8193-45fe-aefd-dfcad5a6ca70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h\" (UID: \"62d7756f-8193-45fe-aefd-dfcad5a6ca70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:20.494547 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.494503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d7756f-8193-45fe-aefd-dfcad5a6ca70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h\" (UID: \"62d7756f-8193-45fe-aefd-dfcad5a6ca70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:20.495209 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.495182 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d7756f-8193-45fe-aefd-dfcad5a6ca70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h\" (UID: \"62d7756f-8193-45fe-aefd-dfcad5a6ca70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:20.675817 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.675723 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:20.816073 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.816023 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h"] Aug 05 18:16:20.817952 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:16:20.817929 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d7756f_8193_45fe_aefd_dfcad5a6ca70.slice/crio-cbaf15d81c7860b03256c0c62e5122369a9759cde3ed998a6912bcf322ea8aac WatchSource:0}: Error finding container cbaf15d81c7860b03256c0c62e5122369a9759cde3ed998a6912bcf322ea8aac: Status 404 returned error can't find the container with id cbaf15d81c7860b03256c0c62e5122369a9759cde3ed998a6912bcf322ea8aac Aug 05 18:16:20.932389 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:20.932307 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" event={"ID":"62d7756f-8193-45fe-aefd-dfcad5a6ca70","Type":"ContainerStarted","Data":"cbaf15d81c7860b03256c0c62e5122369a9759cde3ed998a6912bcf322ea8aac"} Aug 05 18:16:21.941271 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:21.941182 2575 generic.go:356] "Generic (PLEG): container finished" podID="62d7756f-8193-45fe-aefd-dfcad5a6ca70" containerID="d1eb158cd88abd77d93aff12efe8215b3e8a6209bcb5812077d3925c62dceb5c" exitCode=0 Aug 05 18:16:21.941648 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:21.941277 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" event={"ID":"62d7756f-8193-45fe-aefd-dfcad5a6ca70","Type":"ContainerDied","Data":"d1eb158cd88abd77d93aff12efe8215b3e8a6209bcb5812077d3925c62dceb5c"} Aug 05 18:16:23.088021 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.087994 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:23.117869 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.117840 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d7756f-8193-45fe-aefd-dfcad5a6ca70-maas-api-service-ca\") pod \"62d7756f-8193-45fe-aefd-dfcad5a6ca70\" (UID: \"62d7756f-8193-45fe-aefd-dfcad5a6ca70\") " Aug 05 18:16:23.118171 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.118148 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d7756f-8193-45fe-aefd-dfcad5a6ca70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62d7756f-8193-45fe-aefd-dfcad5a6ca70" (UID: "62d7756f-8193-45fe-aefd-dfcad5a6ca70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:16:23.218676 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.218560 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62d7756f-8193-45fe-aefd-dfcad5a6ca70-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:23.962217 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.962176 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" Aug 05 18:16:23.962411 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.962214 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h" event={"ID":"62d7756f-8193-45fe-aefd-dfcad5a6ca70","Type":"ContainerDied","Data":"cbaf15d81c7860b03256c0c62e5122369a9759cde3ed998a6912bcf322ea8aac"} Aug 05 18:16:23.962411 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:23.962268 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbaf15d81c7860b03256c0c62e5122369a9759cde3ed998a6912bcf322ea8aac" Aug 05 18:16:25.222197 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.222160 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9"] Aug 05 18:16:25.222672 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.222404 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" podUID="cd86d37b-133c-4145-8d82-446c89692a31" containerName="payload-pre-processing" containerID="cri-o://cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e" gracePeriod=30 Aug 05 18:16:25.354656 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.354624 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h"] Aug 05 18:16:25.357112 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.357088 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-fb84d0-8976h"] Aug 05 18:16:25.479930 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.479862 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:16:25.541679 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.541637 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqzbg\" (UniqueName: \"kubernetes.io/projected/cd86d37b-133c-4145-8d82-446c89692a31-kube-api-access-xqzbg\") pod \"cd86d37b-133c-4145-8d82-446c89692a31\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " Aug 05 18:16:25.541880 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.541706 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd86d37b-133c-4145-8d82-446c89692a31-plugins-config-volume\") pod \"cd86d37b-133c-4145-8d82-446c89692a31\" (UID: \"cd86d37b-133c-4145-8d82-446c89692a31\") " Aug 05 18:16:25.542037 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.542012 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd86d37b-133c-4145-8d82-446c89692a31-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd86d37b-133c-4145-8d82-446c89692a31" (UID: "cd86d37b-133c-4145-8d82-446c89692a31"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:16:25.543677 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.543654 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd86d37b-133c-4145-8d82-446c89692a31-kube-api-access-xqzbg" (OuterVolumeSpecName: "kube-api-access-xqzbg") pod "cd86d37b-133c-4145-8d82-446c89692a31" (UID: "cd86d37b-133c-4145-8d82-446c89692a31"). InnerVolumeSpecName "kube-api-access-xqzbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:16:25.643407 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.643368 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqzbg\" (UniqueName: \"kubernetes.io/projected/cd86d37b-133c-4145-8d82-446c89692a31-kube-api-access-xqzbg\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:25.643407 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.643401 2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd86d37b-133c-4145-8d82-446c89692a31-plugins-config-volume\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:25.976103 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.976046 2575 generic.go:356] "Generic (PLEG): container finished" podID="cd86d37b-133c-4145-8d82-446c89692a31" containerID="cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e" exitCode=0 Aug 05 18:16:25.976277 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.976109 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" Aug 05 18:16:25.976277 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.976145 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" event={"ID":"cd86d37b-133c-4145-8d82-446c89692a31","Type":"ContainerDied","Data":"cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e"} Aug 05 18:16:25.976277 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.976205 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9" event={"ID":"cd86d37b-133c-4145-8d82-446c89692a31","Type":"ContainerDied","Data":"767bc5fe593aebf3d2526628af3ff1801f163a4042eacc6832a11a5760c96b1f"} Aug 05 18:16:25.976277 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.976234 2575 scope.go:117] "RemoveContainer" containerID="cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e" Aug 05 18:16:25.986378 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.986360 2575 scope.go:117] "RemoveContainer" containerID="cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e" Aug 05 18:16:25.986654 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:16:25.986636 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e\": container with ID starting with cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e not found: ID does not exist" containerID="cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e" Aug 05 18:16:25.986702 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.986663 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e"} err="failed to get container status \"cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e\": rpc error: code = NotFound desc = could not find container \"cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e\": container with ID starting with cf0dee5bf88c253298bfb2e0044fab262359584bd552a278bfd9021d4b43966e not found: ID does not exist" Aug 05 18:16:25.999536 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:25.999513 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9"] Aug 05 18:16:26.006624 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:26.006604 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-fb84d0-5d5b5f946d-lwlv9"] Aug 05 18:16:27.026415 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:27.026380 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62d7756f-8193-45fe-aefd-dfcad5a6ca70" path="/var/lib/kubelet/pods/62d7756f-8193-45fe-aefd-dfcad5a6ca70/volumes" Aug 05 18:16:27.026828 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:27.026768 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd86d37b-133c-4145-8d82-446c89692a31" path="/var/lib/kubelet/pods/cd86d37b-133c-4145-8d82-446c89692a31/volumes" Aug 05 18:16:28.564936 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:28.564898 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:16:35.669744 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.669701 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7"] Aug 05 18:16:35.670174 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.670159 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62d7756f-8193-45fe-aefd-dfcad5a6ca70" containerName="revoke-keys" Aug 05 18:16:35.670231 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.670175 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="62d7756f-8193-45fe-aefd-dfcad5a6ca70" containerName="revoke-keys" Aug 05 18:16:35.670231 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.670210 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd86d37b-133c-4145-8d82-446c89692a31" containerName="payload-pre-processing" Aug 05 18:16:35.670231 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.670217 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd86d37b-133c-4145-8d82-446c89692a31" containerName="payload-pre-processing" Aug 05 18:16:35.670338 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.670292 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="62d7756f-8193-45fe-aefd-dfcad5a6ca70" containerName="revoke-keys" Aug 05 18:16:35.670338 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.670308 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd86d37b-133c-4145-8d82-446c89692a31" containerName="payload-pre-processing" Aug 05 18:16:35.674851 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.674833 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:35.677170 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.677141 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:16:35.679447 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.679420 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7"] Aug 05 18:16:35.742371 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.742333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef479ca6-b8be-40f0-ad89-0431dc89e261-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7\" (UID: \"ef479ca6-b8be-40f0-ad89-0431dc89e261\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:35.843195 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.843157 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef479ca6-b8be-40f0-ad89-0431dc89e261-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7\" (UID: \"ef479ca6-b8be-40f0-ad89-0431dc89e261\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:35.843807 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.843784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef479ca6-b8be-40f0-ad89-0431dc89e261-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7\" (UID: \"ef479ca6-b8be-40f0-ad89-0431dc89e261\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:35.988643 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:35.988556 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:36.129316 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:36.129292 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7"] Aug 05 18:16:36.131349 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:16:36.131321 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef479ca6_b8be_40f0_ad89_0431dc89e261.slice/crio-0113d408399b32258050bc7ef03b1db13ad13f02cf943e50f04cd8cacd320dee WatchSource:0}: Error finding container 0113d408399b32258050bc7ef03b1db13ad13f02cf943e50f04cd8cacd320dee: Status 404 returned error can't find the container with id 0113d408399b32258050bc7ef03b1db13ad13f02cf943e50f04cd8cacd320dee Aug 05 18:16:37.050445 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:37.050408 2575 generic.go:356] "Generic (PLEG): container finished" podID="ef479ca6-b8be-40f0-ad89-0431dc89e261" containerID="71090b9c08097cefc62810383a0c1b46f6fabe41ba66a71ef762b0ec03277d5c" exitCode=0 Aug 05 18:16:37.050445 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:37.050448 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" event={"ID":"ef479ca6-b8be-40f0-ad89-0431dc89e261","Type":"ContainerDied","Data":"71090b9c08097cefc62810383a0c1b46f6fabe41ba66a71ef762b0ec03277d5c"} Aug 05 18:16:37.050887 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:37.050493 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" event={"ID":"ef479ca6-b8be-40f0-ad89-0431dc89e261","Type":"ContainerStarted","Data":"0113d408399b32258050bc7ef03b1db13ad13f02cf943e50f04cd8cacd320dee"} Aug 05 18:16:38.201424 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:38.201396 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:38.268955 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:38.268917 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef479ca6-b8be-40f0-ad89-0431dc89e261-maas-api-service-ca\") pod \"ef479ca6-b8be-40f0-ad89-0431dc89e261\" (UID: \"ef479ca6-b8be-40f0-ad89-0431dc89e261\") " Aug 05 18:16:38.269354 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:38.269331 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef479ca6-b8be-40f0-ad89-0431dc89e261-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef479ca6-b8be-40f0-ad89-0431dc89e261" (UID: "ef479ca6-b8be-40f0-ad89-0431dc89e261"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:16:38.370697 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:38.370604 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef479ca6-b8be-40f0-ad89-0431dc89e261-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:39.066430 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:39.066392 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" Aug 05 18:16:39.066653 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:39.066424 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7" event={"ID":"ef479ca6-b8be-40f0-ad89-0431dc89e261","Type":"ContainerDied","Data":"0113d408399b32258050bc7ef03b1db13ad13f02cf943e50f04cd8cacd320dee"} Aug 05 18:16:39.066653 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:39.066481 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0113d408399b32258050bc7ef03b1db13ad13f02cf943e50f04cd8cacd320dee" Aug 05 18:16:48.868261 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:48.868227 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:16:51.193692 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.193659 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv"] Aug 05 18:16:51.194177 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.193907 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" podUID="0407a903-8ed9-47ed-bdcf-fb91dec3620e" containerName="maas-api" containerID="cri-o://f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f" gracePeriod=30 Aug 05 18:16:51.212405 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.212377 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l"] Aug 05 18:16:51.217192 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.217168 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-4babb1-29765895-zbs6l"] Aug 05 18:16:51.384298 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.384263 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7"] Aug 05 18:16:51.386700 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.386674 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-4babb1-6w2z7"] Aug 05 18:16:51.447984 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.447923 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:16:51.603766 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.603734 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0407a903-8ed9-47ed-bdcf-fb91dec3620e-openshift-service-ca\") pod \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " Aug 05 18:16:51.603969 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.603818 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5pwl\" (UniqueName: \"kubernetes.io/projected/0407a903-8ed9-47ed-bdcf-fb91dec3620e-kube-api-access-j5pwl\") pod \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " Aug 05 18:16:51.603969 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.603955 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0407a903-8ed9-47ed-bdcf-fb91dec3620e-maas-api-tls\") pod \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\" (UID: \"0407a903-8ed9-47ed-bdcf-fb91dec3620e\") " Aug 05 18:16:51.604179 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.604153 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0407a903-8ed9-47ed-bdcf-fb91dec3620e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0407a903-8ed9-47ed-bdcf-fb91dec3620e" (UID: "0407a903-8ed9-47ed-bdcf-fb91dec3620e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:16:51.604288 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.604269 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0407a903-8ed9-47ed-bdcf-fb91dec3620e-openshift-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:51.605912 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.605886 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0407a903-8ed9-47ed-bdcf-fb91dec3620e-kube-api-access-j5pwl" (OuterVolumeSpecName: "kube-api-access-j5pwl") pod "0407a903-8ed9-47ed-bdcf-fb91dec3620e" (UID: "0407a903-8ed9-47ed-bdcf-fb91dec3620e"). InnerVolumeSpecName "kube-api-access-j5pwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:16:51.606020 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.605993 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0407a903-8ed9-47ed-bdcf-fb91dec3620e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0407a903-8ed9-47ed-bdcf-fb91dec3620e" (UID: "0407a903-8ed9-47ed-bdcf-fb91dec3620e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 18:16:51.705651 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.705573 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5pwl\" (UniqueName: \"kubernetes.io/projected/0407a903-8ed9-47ed-bdcf-fb91dec3620e-kube-api-access-j5pwl\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:51.705651 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:51.705601 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0407a903-8ed9-47ed-bdcf-fb91dec3620e-maas-api-tls\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:16:52.153875 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.153841 2575 generic.go:356] "Generic (PLEG): container finished" podID="0407a903-8ed9-47ed-bdcf-fb91dec3620e" containerID="f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f" exitCode=0 Aug 05 18:16:52.154046 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.153886 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" Aug 05 18:16:52.154046 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.153931 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" event={"ID":"0407a903-8ed9-47ed-bdcf-fb91dec3620e","Type":"ContainerDied","Data":"f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f"} Aug 05 18:16:52.154046 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.153984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv" event={"ID":"0407a903-8ed9-47ed-bdcf-fb91dec3620e","Type":"ContainerDied","Data":"2f98af969190475725bb49627f1c59a9e2f3f03264717024a70fe3c1da7a78ef"} Aug 05 18:16:52.154046 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.154023 2575 scope.go:117] "RemoveContainer" containerID="f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f" Aug 05 18:16:52.164289 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.164270 2575 scope.go:117] "RemoveContainer" containerID="f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f" Aug 05 18:16:52.164523 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:16:52.164506 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f\": container with ID starting with f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f not found: ID does not exist" containerID="f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f" Aug 05 18:16:52.164591 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.164531 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f"} err="failed to get container status \"f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f\": rpc error: code = NotFound desc = could not find container \"f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f\": container with ID starting with f3597ffa65405bdaf8e1e7b990511a4a192fb688fa91658b2da6573b5b64259f not found: ID does not exist" Aug 05 18:16:52.174838 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.174811 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv"] Aug 05 18:16:52.178393 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:52.178372 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-4babb1-dbd894bcc-z4gdv"] Aug 05 18:16:53.028699 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:53.028667 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0407a903-8ed9-47ed-bdcf-fb91dec3620e" path="/var/lib/kubelet/pods/0407a903-8ed9-47ed-bdcf-fb91dec3620e/volumes" Aug 05 18:16:53.029114 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:53.028958 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f82df0d-27cf-4f94-8d10-16ada09b3376" path="/var/lib/kubelet/pods/6f82df0d-27cf-4f94-8d10-16ada09b3376/volumes" Aug 05 18:16:53.029252 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:53.029238 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef479ca6-b8be-40f0-ad89-0431dc89e261" path="/var/lib/kubelet/pods/ef479ca6-b8be-40f0-ad89-0431dc89e261/volumes" Aug 05 18:16:56.775689 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:56.775654 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq"] Aug 05 18:16:56.870281 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:16:56.870245 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:17:11.795019 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:11.794969 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" podUID="5967882b-ff2a-4e0c-9baa-2bbd72a87633" containerName="main" containerID="cri-o://1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af" gracePeriod=45 Aug 05 18:17:12.050459 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.050436 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:17:12.098413 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098374 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tmp-dir\") pod \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " Aug 05 18:17:12.098601 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098426 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-model-cache\") pod \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " Aug 05 18:17:12.098601 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098485 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pxxc\" (UniqueName: \"kubernetes.io/projected/5967882b-ff2a-4e0c-9baa-2bbd72a87633-kube-api-access-6pxxc\") pod \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " Aug 05 18:17:12.098601 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098511 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tls-certs\") pod \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " Aug 05 18:17:12.098780 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098610 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-home\") pod \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " Aug 05 18:17:12.098780 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098678 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-dshm\") pod \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\" (UID: \"5967882b-ff2a-4e0c-9baa-2bbd72a87633\") " Aug 05 18:17:12.098780 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098725 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5967882b-ff2a-4e0c-9baa-2bbd72a87633" (UID: "5967882b-ff2a-4e0c-9baa-2bbd72a87633"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:17:12.098780 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098741 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-model-cache" (OuterVolumeSpecName: "model-cache") pod "5967882b-ff2a-4e0c-9baa-2bbd72a87633" (UID: "5967882b-ff2a-4e0c-9baa-2bbd72a87633"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:17:12.098995 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.098954 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-home" (OuterVolumeSpecName: "home") pod "5967882b-ff2a-4e0c-9baa-2bbd72a87633" (UID: "5967882b-ff2a-4e0c-9baa-2bbd72a87633"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:17:12.099179 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.099104 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-home\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:17:12.099179 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.099129 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tmp-dir\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:17:12.099179 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.099144 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-model-cache\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:17:12.101044 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.101012 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5967882b-ff2a-4e0c-9baa-2bbd72a87633" (UID: "5967882b-ff2a-4e0c-9baa-2bbd72a87633"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 18:17:12.101213 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.101180 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5967882b-ff2a-4e0c-9baa-2bbd72a87633-kube-api-access-6pxxc" (OuterVolumeSpecName: "kube-api-access-6pxxc") pod "5967882b-ff2a-4e0c-9baa-2bbd72a87633" (UID: "5967882b-ff2a-4e0c-9baa-2bbd72a87633"). InnerVolumeSpecName "kube-api-access-6pxxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:17:12.101337 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.101316 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-dshm" (OuterVolumeSpecName: "dshm") pod "5967882b-ff2a-4e0c-9baa-2bbd72a87633" (UID: "5967882b-ff2a-4e0c-9baa-2bbd72a87633"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:17:12.199724 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.199664 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5967882b-ff2a-4e0c-9baa-2bbd72a87633-dshm\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:17:12.199724 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.199715 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pxxc\" (UniqueName: \"kubernetes.io/projected/5967882b-ff2a-4e0c-9baa-2bbd72a87633-kube-api-access-6pxxc\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:17:12.199724 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.199726 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5967882b-ff2a-4e0c-9baa-2bbd72a87633-tls-certs\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:17:12.287677 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.287645 2575 generic.go:356] "Generic (PLEG): container finished" podID="5967882b-ff2a-4e0c-9baa-2bbd72a87633" containerID="1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af" exitCode=0 Aug 05 18:17:12.287860 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.287688 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" Aug 05 18:17:12.287860 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.287737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" event={"ID":"5967882b-ff2a-4e0c-9baa-2bbd72a87633","Type":"ContainerDied","Data":"1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af"} Aug 05 18:17:12.287860 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.287788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq" event={"ID":"5967882b-ff2a-4e0c-9baa-2bbd72a87633","Type":"ContainerDied","Data":"a9d0554a1a44a46b5dce851612c44a722a63b24ed5e6a332c24aa3b4ab3a17d7"} Aug 05 18:17:12.287860 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.287802 2575 scope.go:117] "RemoveContainer" containerID="1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af" Aug 05 18:17:12.298163 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.298138 2575 scope.go:117] "RemoveContainer" containerID="1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af" Aug 05 18:17:12.298436 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:17:12.298415 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af\": container with ID starting with 1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af not found: ID does not exist" containerID="1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af" Aug 05 18:17:12.298537 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.298442 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af"} err="failed to get container status \"1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af\": rpc error: code = NotFound desc = could not find container \"1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af\": container with ID starting with 1b51244433b48bfde882f39bc082163c918bb7dcba2c434158c2d368e337d7af not found: ID does not exist" Aug 05 18:17:12.308699 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.308535 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq"] Aug 05 18:17:12.311502 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:12.311479 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-4babb1/ipp-route-4babb1-kserve-5d557b4cd7-xzrnq"] Aug 05 18:17:13.027552 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:17:13.027514 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5967882b-ff2a-4e0c-9baa-2bbd72a87633" path="/var/lib/kubelet/pods/5967882b-ff2a-4e0c-9baa-2bbd72a87633/volumes" Aug 05 18:18:02.816876 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.816791 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4"] Aug 05 18:18:02.817379 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817277 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5967882b-ff2a-4e0c-9baa-2bbd72a87633" containerName="main" Aug 05 18:18:02.817379 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817292 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5967882b-ff2a-4e0c-9baa-2bbd72a87633" containerName="main" Aug 05 18:18:02.817379 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817300 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef479ca6-b8be-40f0-ad89-0431dc89e261" containerName="revoke-keys" Aug 05 18:18:02.817379 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817305 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef479ca6-b8be-40f0-ad89-0431dc89e261" containerName="revoke-keys" Aug 05 18:18:02.817379 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817318 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0407a903-8ed9-47ed-bdcf-fb91dec3620e" containerName="maas-api" Aug 05 18:18:02.817379 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817324 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0407a903-8ed9-47ed-bdcf-fb91dec3620e" containerName="maas-api" Aug 05 18:18:02.817669 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817404 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5967882b-ff2a-4e0c-9baa-2bbd72a87633" containerName="main" Aug 05 18:18:02.817669 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817411 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef479ca6-b8be-40f0-ad89-0431dc89e261" containerName="revoke-keys" Aug 05 18:18:02.817669 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.817420 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0407a903-8ed9-47ed-bdcf-fb91dec3620e" containerName="maas-api" Aug 05 18:18:02.820630 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.820612 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:02.822769 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.822748 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:18:02.827073 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.827029 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4"] Aug 05 18:18:02.894870 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.894821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc62aba1-7a6d-4f72-a083-8ef90db010e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4\" (UID: \"dc62aba1-7a6d-4f72-a083-8ef90db010e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:02.995558 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.995519 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc62aba1-7a6d-4f72-a083-8ef90db010e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4\" (UID: \"dc62aba1-7a6d-4f72-a083-8ef90db010e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:02.996195 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:02.996174 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc62aba1-7a6d-4f72-a083-8ef90db010e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4\" (UID: \"dc62aba1-7a6d-4f72-a083-8ef90db010e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:03.132997 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:03.132908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:03.269630 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:03.269597 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4"] Aug 05 18:18:03.270917 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:18:03.270890 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc62aba1_7a6d_4f72_a083_8ef90db010e9.slice/crio-5240d7419b440a65afea559ac6775feb113877188798fb4796482f684e235b17 WatchSource:0}: Error finding container 5240d7419b440a65afea559ac6775feb113877188798fb4796482f684e235b17: Status 404 returned error can't find the container with id 5240d7419b440a65afea559ac6775feb113877188798fb4796482f684e235b17 Aug 05 18:18:03.609093 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:03.609034 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" event={"ID":"dc62aba1-7a6d-4f72-a083-8ef90db010e9","Type":"ContainerStarted","Data":"95b8cc6879c6caccbbcf5bc30971058da04ad8795be1144c729cd2771c1ad70f"} Aug 05 18:18:03.609270 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:03.609111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" event={"ID":"dc62aba1-7a6d-4f72-a083-8ef90db010e9","Type":"ContainerStarted","Data":"5240d7419b440a65afea559ac6775feb113877188798fb4796482f684e235b17"} Aug 05 18:18:03.624263 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:03.624199 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" podStartSLOduration=1.62417708 podStartE2EDuration="1.62417708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:18:03.623568915 +0000 UTC m=+3521.024797735" watchObservedRunningTime="2026-08-05 18:18:03.62417708 +0000 UTC m=+3521.025405896" Aug 05 18:18:04.616761 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:04.616724 2575 generic.go:356] "Generic (PLEG): container finished" podID="dc62aba1-7a6d-4f72-a083-8ef90db010e9" containerID="95b8cc6879c6caccbbcf5bc30971058da04ad8795be1144c729cd2771c1ad70f" exitCode=0 Aug 05 18:18:04.617205 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:04.616799 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" event={"ID":"dc62aba1-7a6d-4f72-a083-8ef90db010e9","Type":"ContainerDied","Data":"95b8cc6879c6caccbbcf5bc30971058da04ad8795be1144c729cd2771c1ad70f"} Aug 05 18:18:05.761818 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:05.761791 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:05.924008 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:05.923913 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc62aba1-7a6d-4f72-a083-8ef90db010e9-maas-api-service-ca\") pod \"dc62aba1-7a6d-4f72-a083-8ef90db010e9\" (UID: \"dc62aba1-7a6d-4f72-a083-8ef90db010e9\") " Aug 05 18:18:05.924331 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:05.924307 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc62aba1-7a6d-4f72-a083-8ef90db010e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc62aba1-7a6d-4f72-a083-8ef90db010e9" (UID: "dc62aba1-7a6d-4f72-a083-8ef90db010e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:18:06.024694 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:06.024657 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc62aba1-7a6d-4f72-a083-8ef90db010e9-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:18:06.630457 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:06.630423 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" Aug 05 18:18:06.630645 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:06.630458 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4" event={"ID":"dc62aba1-7a6d-4f72-a083-8ef90db010e9","Type":"ContainerDied","Data":"5240d7419b440a65afea559ac6775feb113877188798fb4796482f684e235b17"} Aug 05 18:18:06.630645 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:06.630502 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5240d7419b440a65afea559ac6775feb113877188798fb4796482f684e235b17" Aug 05 18:18:13.235113 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.235081 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt"] Aug 05 18:18:13.235582 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.235303 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" podUID="31fad774-c141-4730-9d58-786652849952" containerName="maas-api" containerID="cri-o://4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e" gracePeriod=30 Aug 05 18:18:13.261548 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.261516 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s"] Aug 05 18:18:13.264582 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.264555 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a83758-29765895-p6x8s"] Aug 05 18:18:13.435731 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.435694 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4"] Aug 05 18:18:13.440143 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.440111 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a83758-4zcs4"] Aug 05 18:18:13.489667 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.489601 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:18:13.597976 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.597942 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fad774-c141-4730-9d58-786652849952-openshift-service-ca\") pod \"31fad774-c141-4730-9d58-786652849952\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " Aug 05 18:18:13.597976 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.597979 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fad774-c141-4730-9d58-786652849952-maas-api-tls\") pod \"31fad774-c141-4730-9d58-786652849952\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " Aug 05 18:18:13.598271 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.598195 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tfkm\" (UniqueName: \"kubernetes.io/projected/31fad774-c141-4730-9d58-786652849952-kube-api-access-6tfkm\") pod \"31fad774-c141-4730-9d58-786652849952\" (UID: \"31fad774-c141-4730-9d58-786652849952\") " Aug 05 18:18:13.598434 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.598404 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31fad774-c141-4730-9d58-786652849952-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31fad774-c141-4730-9d58-786652849952" (UID: "31fad774-c141-4730-9d58-786652849952"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:18:13.598608 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.598585 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fad774-c141-4730-9d58-786652849952-openshift-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:18:13.600931 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.600906 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31fad774-c141-4730-9d58-786652849952-kube-api-access-6tfkm" (OuterVolumeSpecName: "kube-api-access-6tfkm") pod "31fad774-c141-4730-9d58-786652849952" (UID: "31fad774-c141-4730-9d58-786652849952"). InnerVolumeSpecName "kube-api-access-6tfkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:18:13.601031 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.600907 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31fad774-c141-4730-9d58-786652849952-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31fad774-c141-4730-9d58-786652849952" (UID: "31fad774-c141-4730-9d58-786652849952"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 05 18:18:13.675026 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.674990 2575 generic.go:356] "Generic (PLEG): container finished" podID="31fad774-c141-4730-9d58-786652849952" containerID="4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e" exitCode=0 Aug 05 18:18:13.675237 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.675039 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" Aug 05 18:18:13.675237 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.675042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" event={"ID":"31fad774-c141-4730-9d58-786652849952","Type":"ContainerDied","Data":"4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e"} Aug 05 18:18:13.675237 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.675121 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt" event={"ID":"31fad774-c141-4730-9d58-786652849952","Type":"ContainerDied","Data":"abb154ae3df21c28018d285b80b76c5f7066f8dbf35237b48660024f73b136e1"} Aug 05 18:18:13.675237 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.675149 2575 scope.go:117] "RemoveContainer" containerID="4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e" Aug 05 18:18:13.685078 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.685044 2575 scope.go:117] "RemoveContainer" containerID="4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e" Aug 05 18:18:13.685328 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:18:13.685309 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e\": container with ID starting with 4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e not found: ID does not exist" containerID="4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e" Aug 05 18:18:13.685386 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.685336 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e"} err="failed to get container status \"4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e\": rpc error: code = NotFound desc = could not find container \"4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e\": container with ID starting with 4bf9cf4d65b4190dcf0ef0aa4cf2742eb1f847efa729f428ed12a47cfac9ec9e not found: ID does not exist" Aug 05 18:18:13.696748 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.696721 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt"] Aug 05 18:18:13.699235 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.699207 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fad774-c141-4730-9d58-786652849952-maas-api-tls\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:18:13.699357 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.699238 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tfkm\" (UniqueName: \"kubernetes.io/projected/31fad774-c141-4730-9d58-786652849952-kube-api-access-6tfkm\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:18:13.700945 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:13.700924 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a83758-5f9f9479b8-wqqjt"] Aug 05 18:18:15.027040 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:15.027005 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31fad774-c141-4730-9d58-786652849952" path="/var/lib/kubelet/pods/31fad774-c141-4730-9d58-786652849952/volumes" Aug 05 18:18:15.027447 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:15.027344 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6246a525-8ae0-43b0-864a-9512c505fd94" path="/var/lib/kubelet/pods/6246a525-8ae0-43b0-864a-9512c505fd94/volumes" Aug 05 18:18:15.027606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:15.027594 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc62aba1-7a6d-4f72-a083-8ef90db010e9" path="/var/lib/kubelet/pods/dc62aba1-7a6d-4f72-a083-8ef90db010e9/volumes" Aug 05 18:18:15.664081 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:15.664036 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:18:20.494332 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494293 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd"] Aug 05 18:18:20.494807 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494759 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc62aba1-7a6d-4f72-a083-8ef90db010e9" containerName="revoke-keys" Aug 05 18:18:20.494807 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494773 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc62aba1-7a6d-4f72-a083-8ef90db010e9" containerName="revoke-keys" Aug 05 18:18:20.494807 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494790 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31fad774-c141-4730-9d58-786652849952" containerName="maas-api" Aug 05 18:18:20.494807 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494795 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="31fad774-c141-4730-9d58-786652849952" containerName="maas-api" Aug 05 18:18:20.494953 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494862 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc62aba1-7a6d-4f72-a083-8ef90db010e9" containerName="revoke-keys" Aug 05 18:18:20.494953 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.494873 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="31fad774-c141-4730-9d58-786652849952" containerName="maas-api" Aug 05 18:18:20.499307 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.499288 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:20.501459 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.501436 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:18:20.501552 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.501462 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 18:18:20.507310 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.507282 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd"] Aug 05 18:18:20.665380 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.665336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b688db-8e52-421c-9bdf-e892b5186c16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd\" (UID: \"21b688db-8e52-421c-9bdf-e892b5186c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:20.766951 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.766860 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b688db-8e52-421c-9bdf-e892b5186c16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd\" (UID: \"21b688db-8e52-421c-9bdf-e892b5186c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:20.767568 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.767541 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b688db-8e52-421c-9bdf-e892b5186c16-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd\" (UID: \"21b688db-8e52-421c-9bdf-e892b5186c16\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:20.811013 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.810977 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:20.953646 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:20.953617 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd"] Aug 05 18:18:20.955509 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:18:20.955482 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b688db_8e52_421c_9bdf_e892b5186c16.slice/crio-67fb1154f374807435c48cab3b49e025f1e77aac013c17e3eda724c5ed221d3f WatchSource:0}: Error finding container 67fb1154f374807435c48cab3b49e025f1e77aac013c17e3eda724c5ed221d3f: Status 404 returned error can't find the container with id 67fb1154f374807435c48cab3b49e025f1e77aac013c17e3eda724c5ed221d3f Aug 05 18:18:21.725217 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:21.725183 2575 generic.go:356] "Generic (PLEG): container finished" podID="21b688db-8e52-421c-9bdf-e892b5186c16" containerID="9d7f938df42917c89d2ddd0d5160131c4f9e5ca9c6946c71d2b0355cd9c1e6b4" exitCode=0 Aug 05 18:18:21.725629 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:21.725271 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" event={"ID":"21b688db-8e52-421c-9bdf-e892b5186c16","Type":"ContainerDied","Data":"9d7f938df42917c89d2ddd0d5160131c4f9e5ca9c6946c71d2b0355cd9c1e6b4"} Aug 05 18:18:21.725629 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:21.725344 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" event={"ID":"21b688db-8e52-421c-9bdf-e892b5186c16","Type":"ContainerStarted","Data":"67fb1154f374807435c48cab3b49e025f1e77aac013c17e3eda724c5ed221d3f"} Aug 05 18:18:22.874526 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:22.874496 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:22.885919 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:22.885887 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b688db-8e52-421c-9bdf-e892b5186c16-maas-api-service-ca\") pod \"21b688db-8e52-421c-9bdf-e892b5186c16\" (UID: \"21b688db-8e52-421c-9bdf-e892b5186c16\") " Aug 05 18:18:22.886276 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:22.886252 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b688db-8e52-421c-9bdf-e892b5186c16-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21b688db-8e52-421c-9bdf-e892b5186c16" (UID: "21b688db-8e52-421c-9bdf-e892b5186c16"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:18:22.987634 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:22.987521 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b688db-8e52-421c-9bdf-e892b5186c16-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:18:23.738583 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:23.738549 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" Aug 05 18:18:23.738583 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:23.738558 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd" event={"ID":"21b688db-8e52-421c-9bdf-e892b5186c16","Type":"ContainerDied","Data":"67fb1154f374807435c48cab3b49e025f1e77aac013c17e3eda724c5ed221d3f"} Aug 05 18:18:23.738802 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:23.738610 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67fb1154f374807435c48cab3b49e025f1e77aac013c17e3eda724c5ed221d3f" Aug 05 18:18:30.943929 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:30.943883 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm"] Aug 05 18:18:30.946499 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:30.946469 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-3f8e0d-29765895-vzsfm"] Aug 05 18:18:31.027064 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:31.027026 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="176f3f63-4035-4359-ad52-8202be20c561" path="/var/lib/kubelet/pods/176f3f63-4035-4359-ad52-8202be20c561/volumes" Aug 05 18:18:31.116480 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:31.116435 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd"] Aug 05 18:18:31.124560 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:31.124518 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3f8e0d-wbscd"] Aug 05 18:18:33.027319 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:33.027288 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21b688db-8e52-421c-9bdf-e892b5186c16" path="/var/lib/kubelet/pods/21b688db-8e52-421c-9bdf-e892b5186c16/volumes" Aug 05 18:18:33.172640 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:33.172604 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:18:38.171081 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.171017 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph"] Aug 05 18:18:38.171656 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.171642 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b688db-8e52-421c-9bdf-e892b5186c16" containerName="revoke-keys" Aug 05 18:18:38.171733 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.171663 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b688db-8e52-421c-9bdf-e892b5186c16" containerName="revoke-keys" Aug 05 18:18:38.171798 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.171766 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b688db-8e52-421c-9bdf-e892b5186c16" containerName="revoke-keys" Aug 05 18:18:38.174894 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.174873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:38.177322 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.177294 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 05 18:18:38.177442 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.177399 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bz9qt\"" Aug 05 18:18:38.181602 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.181579 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph"] Aug 05 18:18:38.232485 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.232449 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03845b8a-50bf-4dd6-ac14-ccac885453a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph\" (UID: \"03845b8a-50bf-4dd6-ac14-ccac885453a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:38.333454 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.333417 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03845b8a-50bf-4dd6-ac14-ccac885453a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph\" (UID: \"03845b8a-50bf-4dd6-ac14-ccac885453a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:38.334039 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.334018 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03845b8a-50bf-4dd6-ac14-ccac885453a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph\" (UID: \"03845b8a-50bf-4dd6-ac14-ccac885453a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:38.486644 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.486558 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:38.620347 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.620319 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph"] Aug 05 18:18:38.622224 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:18:38.622195 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03845b8a_50bf_4dd6_ac14_ccac885453a1.slice/crio-aa6998905b89fe073921d2b279850de6fc42556831b9b8dae8e15c5e2cfd1cd8 WatchSource:0}: Error finding container aa6998905b89fe073921d2b279850de6fc42556831b9b8dae8e15c5e2cfd1cd8: Status 404 returned error can't find the container with id aa6998905b89fe073921d2b279850de6fc42556831b9b8dae8e15c5e2cfd1cd8 Aug 05 18:18:38.834032 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:38.833997 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" event={"ID":"03845b8a-50bf-4dd6-ac14-ccac885453a1","Type":"ContainerStarted","Data":"aa6998905b89fe073921d2b279850de6fc42556831b9b8dae8e15c5e2cfd1cd8"} Aug 05 18:18:39.842897 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:39.842855 2575 generic.go:356] "Generic (PLEG): container finished" podID="03845b8a-50bf-4dd6-ac14-ccac885453a1" containerID="5143d05498a5093822486ed9f8222f253abe100696674dd114947a3f82856559" exitCode=0 Aug 05 18:18:39.843347 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:39.842938 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" event={"ID":"03845b8a-50bf-4dd6-ac14-ccac885453a1","Type":"ContainerDied","Data":"5143d05498a5093822486ed9f8222f253abe100696674dd114947a3f82856559"} Aug 05 18:18:40.977540 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:40.977514 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:41.058750 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:41.058714 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03845b8a-50bf-4dd6-ac14-ccac885453a1-maas-api-service-ca\") pod \"03845b8a-50bf-4dd6-ac14-ccac885453a1\" (UID: \"03845b8a-50bf-4dd6-ac14-ccac885453a1\") " Aug 05 18:18:41.059081 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:41.059037 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03845b8a-50bf-4dd6-ac14-ccac885453a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03845b8a-50bf-4dd6-ac14-ccac885453a1" (UID: "03845b8a-50bf-4dd6-ac14-ccac885453a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:18:41.159997 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:41.159911 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03845b8a-50bf-4dd6-ac14-ccac885453a1-maas-api-service-ca\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:18:41.856289 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:41.856253 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" event={"ID":"03845b8a-50bf-4dd6-ac14-ccac885453a1","Type":"ContainerDied","Data":"aa6998905b89fe073921d2b279850de6fc42556831b9b8dae8e15c5e2cfd1cd8"} Aug 05 18:18:41.856477 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:41.856302 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa6998905b89fe073921d2b279850de6fc42556831b9b8dae8e15c5e2cfd1cd8" Aug 05 18:18:41.856477 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:41.856263 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph" Aug 05 18:18:48.626979 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:48.626934 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7"] Aug 05 18:18:48.631101 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:48.631075 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-30613e-29765895-r24p7"] Aug 05 18:18:48.806770 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:48.806735 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph"] Aug 05 18:18:48.809040 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:48.809014 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-30613e-gm4ph"] Aug 05 18:18:49.027485 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:49.027413 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03845b8a-50bf-4dd6-ac14-ccac885453a1" path="/var/lib/kubelet/pods/03845b8a-50bf-4dd6-ac14-ccac885453a1/volumes" Aug 05 18:18:49.027723 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:49.027707 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353aaaab-f9ce-4cdf-ae11-915f6ca04861" path="/var/lib/kubelet/pods/353aaaab-f9ce-4cdf-ae11-915f6ca04861/volumes" Aug 05 18:18:50.861895 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:50.861859 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-sq7qh"] Aug 05 18:18:55.529470 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:55.529435 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9"] Aug 05 18:18:55.529912 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:55.529655 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" containerID="cri-o://bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0" gracePeriod=30 Aug 05 18:18:56.217943 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:18:56.217909 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c77fb47d4-cskr9_fb6cdc6d-3d42-4539-bfaa-f9268b517fef/authorino/0.log" Aug 05 18:19:00.792360 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.792328 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:19:00.947626 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947596 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-token\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.947788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947634 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-envoy\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.947788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947659 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d703899-6f4a-4711-ab32-7384a51cb8ca-istiod-ca-cert\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.947788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947703 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-credential-socket\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.947788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947729 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-certs\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.947788 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947771 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-data\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.948100 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947805 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8btn5\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-kube-api-access-8btn5\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.948100 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947896 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-podinfo\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.948100 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.947970 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-socket\") pod \"4d703899-6f4a-4711-ab32-7384a51cb8ca\" (UID: \"4d703899-6f4a-4711-ab32-7384a51cb8ca\") " Aug 05 18:19:00.948278 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948124 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:19:00.948278 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948135 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:19:00.948278 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948206 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d703899-6f4a-4711-ab32-7384a51cb8ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 05 18:19:00.948444 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948333 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d703899-6f4a-4711-ab32-7384a51cb8ca-istiod-ca-cert\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:00.948444 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948345 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:19:00.948444 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948358 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-credential-socket\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:00.948444 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948372 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-data\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:00.948444 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.948412 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:19:00.949929 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.949897 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 05 18:19:00.950036 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.949963 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:19:00.950036 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.949983 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-kube-api-access-8btn5" (OuterVolumeSpecName: "kube-api-access-8btn5") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "kube-api-access-8btn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 05 18:19:00.950166 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.950108 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d703899-6f4a-4711-ab32-7384a51cb8ca" (UID: "4d703899-6f4a-4711-ab32-7384a51cb8ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 05 18:19:00.996499 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.996468 2575 generic.go:356] "Generic (PLEG): container finished" podID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerID="bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0" exitCode=0 Aug 05 18:19:00.996637 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.996524 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" Aug 05 18:19:00.996637 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.996537 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" event={"ID":"4d703899-6f4a-4711-ab32-7384a51cb8ca","Type":"ContainerDied","Data":"bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0"} Aug 05 18:19:00.996637 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.996601 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9" event={"ID":"4d703899-6f4a-4711-ab32-7384a51cb8ca","Type":"ContainerDied","Data":"ef3ac13d809c2074db942737a90872264fc7ff9b31daca2f08827a3364952827"} Aug 05 18:19:00.996637 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:00.996623 2575 scope.go:117] "RemoveContainer" containerID="bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0" Aug 05 18:19:01.006441 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.006416 2575 scope.go:117] "RemoveContainer" containerID="daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc" Aug 05 18:19:01.014859 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.014841 2575 scope.go:117] "RemoveContainer" containerID="bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0" Aug 05 18:19:01.015157 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:19:01.015133 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0\": container with ID starting with bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0 not found: ID does not exist" containerID="bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0" Aug 05 18:19:01.015236 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.015166 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0"} err="failed to get container status \"bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0\": rpc error: code = NotFound desc = could not find container \"bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0\": container with ID starting with bed7c5bc6858b2fa025d06766094cd72f03e2a762a712910696f58b98708f6d0 not found: ID does not exist" Aug 05 18:19:01.015236 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.015182 2575 scope.go:117] "RemoveContainer" containerID="daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc" Aug 05 18:19:01.015447 ip-10-0-143-58 kubenswrapper[2575]: E0805 18:19:01.015419 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc\": container with ID starting with daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc not found: ID does not exist" containerID="daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc" Aug 05 18:19:01.015517 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.015454 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc"} err="failed to get container status \"daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc\": rpc error: code = NotFound desc = could not find container \"daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc\": container with ID starting with daebb627345768781e14576ab03800a104c3138d0db54b77bc18c56595db58fc not found: ID does not exist" Aug 05 18:19:01.018437 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.018414 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9"] Aug 05 18:19:01.027027 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.027004 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-30613e-openshift-default-85b8666d56-6vtm9"] Aug 05 18:19:01.049535 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.049517 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-token\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:01.049606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.049538 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-envoy\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:01.049606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.049547 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-certs\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:01.049606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.049555 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8btn5\" (UniqueName: \"kubernetes.io/projected/4d703899-6f4a-4711-ab32-7384a51cb8ca-kube-api-access-8btn5\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:01.049606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.049565 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d703899-6f4a-4711-ab32-7384a51cb8ca-istio-podinfo\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:01.049606 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:01.049574 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d703899-6f4a-4711-ab32-7384a51cb8ca-workload-socket\") on node \"ip-10-0-143-58.ec2.internal\" DevicePath \"\"" Aug 05 18:19:02.251398 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:02.251368 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-27rsr_98a5356c-a479-420a-9906-5c9f54e22d91/copy-manifests/0.log" Aug 05 18:19:02.257535 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:02.257507 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-27rsr_98a5356c-a479-420a-9906-5c9f54e22d91/manager/0.log" Aug 05 18:19:03.026936 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:03.026905 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" path="/var/lib/kubelet/pods/4d703899-6f4a-4711-ab32-7384a51cb8ca/volumes" Aug 05 18:19:03.110416 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:03.110394 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29765880-zshdx_598c18c4-6f79-46af-8006-0e98777be0b9/cleanup/0.log" Aug 05 18:19:03.221297 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:03.221266 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29765895-p2psx_02f8f6fd-0682-413c-8e7c-a8f445526a8d/cleanup/0.log" Aug 05 18:19:04.143833 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:04.143800 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c77fb47d4-cskr9_fb6cdc6d-3d42-4539-bfaa-f9268b517fef/authorino/0.log" Aug 05 18:19:04.388208 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:04.388173 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-2tnj9_387b5967-f81f-4684-9172-b43e9ade410d/manager/0.log" Aug 05 18:19:04.503994 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:04.503896 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-w7rv8_60f7b3b1-811f-4bd0-b4db-61d655f014ed/kuadrant-console-plugin/0.log" Aug 05 18:19:04.741426 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:04.741396 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-sq7qh_76f17861-853b-4a7f-8e21-267baf2dd452/limitador/0.log" Aug 05 18:19:05.568626 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:05.568596 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hvhzg_496be72c-9be6-4d1a-a7ce-0f05028363a9/kube-auth-proxy/0.log" Aug 05 18:19:12.823149 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823113 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f9829/must-gather-bfvhs"] Aug 05 18:19:12.823641 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823508 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" Aug 05 18:19:12.823641 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823521 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" Aug 05 18:19:12.823641 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823542 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03845b8a-50bf-4dd6-ac14-ccac885453a1" containerName="revoke-keys" Aug 05 18:19:12.823641 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823548 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="03845b8a-50bf-4dd6-ac14-ccac885453a1" containerName="revoke-keys" Aug 05 18:19:12.823641 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823612 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" Aug 05 18:19:12.823641 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823625 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="03845b8a-50bf-4dd6-ac14-ccac885453a1" containerName="revoke-keys" Aug 05 18:19:12.823840 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823691 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" Aug 05 18:19:12.823840 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823696 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" Aug 05 18:19:12.823840 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.823753 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d703899-6f4a-4711-ab32-7384a51cb8ca" containerName="istio-proxy" Aug 05 18:19:12.827030 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.827007 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:12.829105 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.829036 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f9829\"/\"default-dockercfg-6655g\"" Aug 05 18:19:12.829105 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.829083 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f9829\"/\"kube-root-ca.crt\"" Aug 05 18:19:12.829105 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.829094 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f9829\"/\"openshift-service-ca.crt\"" Aug 05 18:19:12.834756 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.834309 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f9829/must-gather-bfvhs"] Aug 05 18:19:12.856007 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.855970 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhp6v\" (UniqueName: \"kubernetes.io/projected/186d295a-fdcd-497c-bacd-51015afb0cb6-kube-api-access-lhp6v\") pod \"must-gather-bfvhs\" (UID: \"186d295a-fdcd-497c-bacd-51015afb0cb6\") " pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:12.856198 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.856033 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186d295a-fdcd-497c-bacd-51015afb0cb6-must-gather-output\") pod \"must-gather-bfvhs\" (UID: \"186d295a-fdcd-497c-bacd-51015afb0cb6\") " pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:12.956724 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.956687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhp6v\" (UniqueName: \"kubernetes.io/projected/186d295a-fdcd-497c-bacd-51015afb0cb6-kube-api-access-lhp6v\") pod \"must-gather-bfvhs\" (UID: \"186d295a-fdcd-497c-bacd-51015afb0cb6\") " pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:12.956924 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.956755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186d295a-fdcd-497c-bacd-51015afb0cb6-must-gather-output\") pod \"must-gather-bfvhs\" (UID: \"186d295a-fdcd-497c-bacd-51015afb0cb6\") " pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:12.957185 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.957161 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186d295a-fdcd-497c-bacd-51015afb0cb6-must-gather-output\") pod \"must-gather-bfvhs\" (UID: \"186d295a-fdcd-497c-bacd-51015afb0cb6\") " pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:12.966687 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:12.966665 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhp6v\" (UniqueName: \"kubernetes.io/projected/186d295a-fdcd-497c-bacd-51015afb0cb6-kube-api-access-lhp6v\") pod \"must-gather-bfvhs\" (UID: \"186d295a-fdcd-497c-bacd-51015afb0cb6\") " pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:13.136968 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:13.136886 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f9829/must-gather-bfvhs" Aug 05 18:19:13.271991 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:13.271965 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f9829/must-gather-bfvhs"] Aug 05 18:19:13.273816 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:19:13.273787 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod186d295a_fdcd_497c_bacd_51015afb0cb6.slice/crio-16df00b5ab70b837aaf91cf68179f4c028e40c17d054ae16df220f38c2315711 WatchSource:0}: Error finding container 16df00b5ab70b837aaf91cf68179f4c028e40c17d054ae16df220f38c2315711: Status 404 returned error can't find the container with id 16df00b5ab70b837aaf91cf68179f4c028e40c17d054ae16df220f38c2315711 Aug 05 18:19:13.276173 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:13.276152 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 05 18:19:14.086860 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:14.086816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/must-gather-bfvhs" event={"ID":"186d295a-fdcd-497c-bacd-51015afb0cb6","Type":"ContainerStarted","Data":"16df00b5ab70b837aaf91cf68179f4c028e40c17d054ae16df220f38c2315711"} Aug 05 18:19:15.097936 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:15.097843 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/must-gather-bfvhs" event={"ID":"186d295a-fdcd-497c-bacd-51015afb0cb6","Type":"ContainerStarted","Data":"aeb17eeac9a25b1a8d5d2fe6c91b2e0994fa46dedb6ed9c79f16aa585993c40c"} Aug 05 18:19:15.097936 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:15.097904 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/must-gather-bfvhs" event={"ID":"186d295a-fdcd-497c-bacd-51015afb0cb6","Type":"ContainerStarted","Data":"100c1a2eed011ee3ee226d938b154a0780d5b872290992b3ea157570a016832e"} Aug 05 18:19:15.111927 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:15.111868 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f9829/must-gather-bfvhs" podStartSLOduration=2.352860527 podStartE2EDuration="3.111848844s" totalImagesPullingTime="758.988317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:19:15.11098328 +0000 UTC m=+3592.512212096" watchObservedRunningTime="2026-08-05 18:19:15.111848844 +0000 UTC m=+3592.513077658" Aug 05 18:19:15.584871 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:15.584838 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dpb7v_22bf72ca-6129-4438-b017-59817d913812/global-pull-secret-syncer/0.log" Aug 05 18:19:15.760121 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:15.760074 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ffknk_d41c9d97-6ed4-4c96-9587-f20c375cd918/konnectivity-agent/0.log" Aug 05 18:19:15.872600 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:15.872514 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-58.ec2.internal_47f3b5ee5e09766822cedb87a0da628d/haproxy/0.log" Aug 05 18:19:21.120175 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:21.119908 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c77fb47d4-cskr9_fb6cdc6d-3d42-4539-bfaa-f9268b517fef/authorino/0.log" Aug 05 18:19:21.198257 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:21.198226 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-2tnj9_387b5967-f81f-4684-9172-b43e9ade410d/manager/0.log" Aug 05 18:19:21.230372 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:21.230343 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-w7rv8_60f7b3b1-811f-4bd0-b4db-61d655f014ed/kuadrant-console-plugin/0.log" Aug 05 18:19:21.392971 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:21.392884 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-sq7qh_76f17861-853b-4a7f-8e21-267baf2dd452/limitador/0.log" Aug 05 18:19:22.834242 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:22.834203 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/alertmanager/0.log" Aug 05 18:19:22.861113 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:22.861080 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/config-reloader/0.log" Aug 05 18:19:22.888112 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:22.888083 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/kube-rbac-proxy-web/0.log" Aug 05 18:19:22.910322 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:22.910280 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/kube-rbac-proxy/0.log" Aug 05 18:19:22.948432 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:22.948399 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/kube-rbac-proxy-metric/0.log" Aug 05 18:19:22.978239 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:22.978202 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/prom-label-proxy/0.log" Aug 05 18:19:23.003534 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.003481 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c7145719-1731-469b-af9d-9d8decda5652/init-config-reloader/0.log" Aug 05 18:19:23.079794 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.079761 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-g64gp_96c12aea-18c7-45cc-979f-f15b9b968020/kube-state-metrics/0.log" Aug 05 18:19:23.119096 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.119001 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-g64gp_96c12aea-18c7-45cc-979f-f15b9b968020/kube-rbac-proxy-main/0.log" Aug 05 18:19:23.149983 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.149954 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-g64gp_96c12aea-18c7-45cc-979f-f15b9b968020/kube-rbac-proxy-self/0.log" Aug 05 18:19:23.184434 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.184395 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-d54cd6548-xbhpt_257c259a-c957-4373-a341-339925119047/metrics-server/0.log" Aug 05 18:19:23.425812 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.425736 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rl6rc_a6982795-4770-45f8-90e8-ead2e4c904e5/node-exporter/0.log" Aug 05 18:19:23.450326 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.450276 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rl6rc_a6982795-4770-45f8-90e8-ead2e4c904e5/kube-rbac-proxy/0.log" Aug 05 18:19:23.474098 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.474020 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:19:23.482845 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.482815 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rl6rc_a6982795-4770-45f8-90e8-ead2e4c904e5/init-textfile/0.log" Aug 05 18:19:23.483829 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.483807 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:19:23.486540 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.486514 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:19:23.493467 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.493447 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d2f2g_5b8acaa6-4e18-4d02-8f03-d7c62d79b618/ovn-acl-logging/0.log" Aug 05 18:19:23.514498 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.514466 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-ljhd5_ee5b1533-ac9b-404e-bf8a-920d08e2c0bb/kube-rbac-proxy-main/0.log" Aug 05 18:19:23.539071 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.539031 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-ljhd5_ee5b1533-ac9b-404e-bf8a-920d08e2c0bb/kube-rbac-proxy-self/0.log" Aug 05 18:19:23.563425 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.563395 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-ljhd5_ee5b1533-ac9b-404e-bf8a-920d08e2c0bb/openshift-state-metrics/0.log" Aug 05 18:19:23.839816 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.839783 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-jf5cl_241e6e0c-371e-4d2f-a8c2-77c7fbf2c145/prometheus-operator-admission-webhook/0.log" Aug 05 18:19:23.876098 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.876067 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8d66bd447-22gvw_ff3aeea5-e44e-452a-8b3b-be1330ffa857/telemeter-client/0.log" Aug 05 18:19:23.901262 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.901233 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8d66bd447-22gvw_ff3aeea5-e44e-452a-8b3b-be1330ffa857/reload/0.log" Aug 05 18:19:23.924389 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.924356 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8d66bd447-22gvw_ff3aeea5-e44e-452a-8b3b-be1330ffa857/kube-rbac-proxy/0.log" Aug 05 18:19:23.957424 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.957390 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d496b45bc-thc85_e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396/thanos-query/0.log" Aug 05 18:19:23.981820 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:23.981788 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d496b45bc-thc85_e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396/kube-rbac-proxy-web/0.log" Aug 05 18:19:24.004535 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:24.004507 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d496b45bc-thc85_e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396/kube-rbac-proxy/0.log" Aug 05 18:19:24.023656 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:24.023626 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d496b45bc-thc85_e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396/prom-label-proxy/0.log" Aug 05 18:19:24.047986 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:24.047891 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d496b45bc-thc85_e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396/kube-rbac-proxy-rules/0.log" Aug 05 18:19:24.072434 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:24.072403 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d496b45bc-thc85_e85cee89-f2f7-4c6e-b7d1-9f3ba1f83396/kube-rbac-proxy-metrics/0.log" Aug 05 18:19:26.031045 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.030962 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/2.log" Aug 05 18:19:26.037266 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.037235 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-mxs75_451a1bd3-fbed-42c5-a04e-45a6befe04f1/console-operator/3.log" Aug 05 18:19:26.228818 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.228785 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx"] Aug 05 18:19:26.231988 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.231964 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.242369 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.242335 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx"] Aug 05 18:19:26.309194 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.309157 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-proc\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.309500 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.309479 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-sys\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.309689 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.309670 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmjzb\" (UniqueName: \"kubernetes.io/projected/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-kube-api-access-xmjzb\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.309814 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.309800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-podres\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.310014 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.309998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-lib-modules\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.410878 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.410834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-lib-modules\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411107 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.410951 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-proc\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411107 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411039 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-sys\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411107 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411079 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-lib-modules\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411303 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411106 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-proc\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411303 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411155 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-sys\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411303 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411190 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmjzb\" (UniqueName: \"kubernetes.io/projected/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-kube-api-access-xmjzb\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411303 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-podres\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.411522 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.411398 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-podres\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.418711 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.418672 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmjzb\" (UniqueName: \"kubernetes.io/projected/00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d-kube-api-access-xmjzb\") pod \"perf-node-gather-daemonset-f2lpx\" (UID: \"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d\") " pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.547550 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.547508 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:26.612213 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.612179 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f5668dc8d-d28w2_18c99278-a570-4aa6-8820-1ad239713bb2/console/0.log" Aug 05 18:19:26.732585 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:26.732516 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx"] Aug 05 18:19:26.736536 ip-10-0-143-58 kubenswrapper[2575]: W0805 18:19:26.736501 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod00dfb139_d0b3_4c9e_8bc8_cdf39b7c133d.slice/crio-d6a92665f1313652bf0abe9f340f1ae6994197e573cd58d8cbd024c5e174a0a4 WatchSource:0}: Error finding container d6a92665f1313652bf0abe9f340f1ae6994197e573cd58d8cbd024c5e174a0a4: Status 404 returned error can't find the container with id d6a92665f1313652bf0abe9f340f1ae6994197e573cd58d8cbd024c5e174a0a4 Aug 05 18:19:27.223722 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:27.223621 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" event={"ID":"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d","Type":"ContainerStarted","Data":"b170f6f9a77236b501cb53271cecc8ecef365f4dfdd9fa5a5f07589c1cdffa59"} Aug 05 18:19:27.223722 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:27.223680 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" event={"ID":"00dfb139-d0b3-4c9e-8bc8-cdf39b7c133d","Type":"ContainerStarted","Data":"d6a92665f1313652bf0abe9f340f1ae6994197e573cd58d8cbd024c5e174a0a4"} Aug 05 18:19:27.224273 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:27.223789 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:27.330276 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:27.330243 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-xkpwp_a964634b-ad74-4dd6-9cc1-13361d691cbb/volume-data-source-validator/0.log" Aug 05 18:19:28.390037 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:28.390007 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cb75m_b46f48a8-01a7-4ba2-bcc3-b5892a16caf7/dns/0.log" Aug 05 18:19:28.410277 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:28.410245 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cb75m_b46f48a8-01a7-4ba2-bcc3-b5892a16caf7/kube-rbac-proxy/0.log" Aug 05 18:19:28.502919 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:28.502890 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9gz9l_94edb85c-3031-405a-bf80-76c2d0e4ed88/dns-node-resolver/0.log" Aug 05 18:19:29.104563 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:29.104534 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9cf2h_838305c7-2d97-492f-bf25-e2f8f485ae5d/node-ca/0.log" Aug 05 18:19:30.278884 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:30.278840 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hvhzg_496be72c-9be6-4d1a-a7ce-0f05028363a9/kube-auth-proxy/0.log" Aug 05 18:19:31.272447 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:31.272391 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p8tz8_298eeb1d-3ed1-407f-990f-08b90ccd84f5/serve-healthcheck-canary/0.log" Aug 05 18:19:31.775336 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:31.774861 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-75ch2_36009ed7-f9b0-48d3-920e-3d6d452d18e8/insights-operator/0.log" Aug 05 18:19:31.777676 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:31.777622 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-75ch2_36009ed7-f9b0-48d3-920e-3d6d452d18e8/insights-operator/1.log" Aug 05 18:19:31.869473 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:31.869441 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g69mz_56167886-dcec-4258-a9f9-286f063b0887/kube-rbac-proxy/0.log" Aug 05 18:19:31.891981 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:31.891948 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g69mz_56167886-dcec-4258-a9f9-286f063b0887/exporter/0.log" Aug 05 18:19:31.914728 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:31.914688 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g69mz_56167886-dcec-4258-a9f9-286f063b0887/extractor/0.log" Aug 05 18:19:33.244163 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:33.244130 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" Aug 05 18:19:33.261307 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:33.261250 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f9829/perf-node-gather-daemonset-f2lpx" podStartSLOduration=7.2612349179999995 podStartE2EDuration="7.261234918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 18:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:19:27.238001982 +0000 UTC m=+3604.639230799" watchObservedRunningTime="2026-08-05 18:19:33.261234918 +0000 UTC m=+3610.662463733" Aug 05 18:19:34.887789 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:34.887747 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-27rsr_98a5356c-a479-420a-9906-5c9f54e22d91/manager/0.log" Aug 05 18:19:34.909278 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:34.909243 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58c579b484-27rsr_98a5356c-a479-420a-9906-5c9f54e22d91/copy-manifests/0.log" Aug 05 18:19:36.527832 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:36.527748 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57fb49d769-xk5sw_dffd07d5-019b-44db-9707-01ca6c262890/manager/0.log" Aug 05 18:19:36.553092 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:36.553041 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5rhfl_b968d3ee-dc99-4362-a7fb-dd9becdd1694/openshift-lws-operator/0.log" Aug 05 18:19:36.890435 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:36.890394 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hvhzg_496be72c-9be6-4d1a-a7ce-0f05028363a9/kube-auth-proxy/0.log" Aug 05 18:19:38.599920 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:38.599876 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c99fc649-hvhzg_496be72c-9be6-4d1a-a7ce-0f05028363a9/kube-auth-proxy/0.log" Aug 05 18:19:41.992141 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:41.992098 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-b8lxd_fe5d030a-f113-49df-b653-5673455bf156/kube-storage-version-migrator-operator/1.log" Aug 05 18:19:41.993924 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:41.993893 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-b8lxd_fe5d030a-f113-49df-b653-5673455bf156/kube-storage-version-migrator-operator/0.log" Aug 05 18:19:43.082913 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.082835 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9799r_2c72aae8-03ad-49f4-9b8b-ad3c0f87d9f3/kube-multus/0.log" Aug 05 18:19:43.270608 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.270583 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/kube-multus-additional-cni-plugins/0.log" Aug 05 18:19:43.293018 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.292988 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/egress-router-binary-copy/0.log" Aug 05 18:19:43.312752 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.312725 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/cni-plugins/0.log" Aug 05 18:19:43.332020 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.331991 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/bond-cni-plugin/0.log" Aug 05 18:19:43.354222 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.354132 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/routeoverride-cni/0.log" Aug 05 18:19:43.384442 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.384414 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/whereabouts-cni-bincopy/0.log" Aug 05 18:19:43.406970 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.406943 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9mv62_541ddbda-527f-4e49-8236-61e81af9f815/whereabouts-cni/0.log" Aug 05 18:19:43.693454 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.693369 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2m4bd_cd322441-fd87-4da5-babe-874dda982d83/network-metrics-daemon/0.log" Aug 05 18:19:43.713489 ip-10-0-143-58 kubenswrapper[2575]: I0805 18:19:43.713460 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2m4bd_cd322441-fd87-4da5-babe-874dda982d83/kube-rbac-proxy/0.log"