Jul 27 16:54:30.693766 ip-10-0-135-54 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 27 16:54:30.693914 ip-10-0-135-54 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 27 16:54:30.694007 ip-10-0-135-54 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 27 16:54:30.694321 ip-10-0-135-54 systemd[1]: Failed to start Kubernetes Kubelet. Jul 27 16:54:40.942886 ip-10-0-135-54 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 27 16:54:40.942905 ip-10-0-135-54 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8e2de6284e47477a99a08de9f63d8107 -- Jul 27 16:56:33.619682 ip-10-0-135-54 systemd[1]: Starting Kubernetes Kubelet... Jul 27 16:56:33.946518 ip-10-0-135-54 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 16:56:33.946518 ip-10-0-135-54 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 27 16:56:33.946518 ip-10-0-135-54 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 16:56:33.946518 ip-10-0-135-54 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 27 16:56:33.946518 ip-10-0-135-54 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 16:56:33.947392 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.947287 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 27 16:56:33.951516 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951499 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 16:56:33.951516 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951516 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951522 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951525 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951529 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951531 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951534 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951537 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951541 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951545 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951548 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951551 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951554 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951557 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951561 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951564 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951568 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951571 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951573 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951576 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 16:56:33.951579 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951579 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951581 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951584 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951587 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951590 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951593 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951596 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951598 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951601 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951604 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951607 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951609 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951612 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951615 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951617 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951620 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951623 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951625 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951629 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951631 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 16:56:33.952084 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951634 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951636 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951639 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951642 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951644 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951647 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951649 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951653 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951655 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951658 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951660 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951663 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951665 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951667 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951671 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951674 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951677 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951679 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951682 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951685 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 16:56:33.952608 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951688 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951690 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951693 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951696 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951699 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951701 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951704 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951706 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951710 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951714 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951716 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951719 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951722 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951725 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951727 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951730 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951732 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951735 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951738 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 16:56:33.953096 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951741 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951743 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951746 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951748 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951751 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951754 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951756 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951759 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951763 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951766 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951769 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951773 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951776 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951779 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951782 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951786 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951789 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951791 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951794 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 16:56:33.953570 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.951797 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952531 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952539 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952542 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952544 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952548 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952552 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952554 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952557 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952560 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952562 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952565 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952567 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952570 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952573 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952575 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952578 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952580 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952583 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952587 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 16:56:33.954034 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952590 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952593 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952596 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952598 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952601 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952603 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952606 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952609 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952611 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952614 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952617 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952619 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952622 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952625 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952627 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952630 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952632 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952635 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952638 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952641 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 16:56:33.954543 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952643 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952646 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952648 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952651 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952653 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952656 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952658 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952661 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952663 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952666 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952668 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952671 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952674 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952677 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952679 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952682 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952685 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952687 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952690 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 16:56:33.955043 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952693 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952695 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952698 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952700 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952703 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952705 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952708 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952710 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952713 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952716 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952719 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952721 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952724 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952728 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952730 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952733 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952735 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952738 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952742 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 16:56:33.955522 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952745 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952748 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952752 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952754 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952758 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952762 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952765 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952768 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952771 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952774 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952777 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952780 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952783 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952786 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952788 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952792 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952796 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952798 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952801 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952804 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 16:56:33.956011 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952806 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.952809 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953747 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953753 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953757 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953760 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953764 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953767 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953770 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953773 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953775 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953778 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953781 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953784 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953787 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953789 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953792 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953794 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953797 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953800 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 16:56:33.956523 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953803 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953805 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953808 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953824 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953828 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953831 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953835 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953839 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953842 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953845 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953848 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953851 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953854 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953857 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953861 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953864 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953866 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953869 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953872 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 16:56:33.957038 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953875 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953877 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953880 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953882 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953885 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953887 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953890 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953892 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953895 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953897 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953900 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953903 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953906 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953909 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953911 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953914 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953917 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953919 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953922 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953924 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 16:56:33.957531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953927 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953930 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953932 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953935 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953937 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953940 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953942 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953945 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953948 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953952 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953955 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953957 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953960 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953963 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953965 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953968 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953970 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953973 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953975 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953978 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 16:56:33.958015 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953980 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953983 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953985 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953990 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953993 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953996 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.953999 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954001 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954004 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954007 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954011 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954014 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954017 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954020 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954023 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954025 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954028 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954030 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954033 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 16:56:33.958539 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954036 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954039 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954042 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954111 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954124 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954132 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954137 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954142 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954146 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954152 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954156 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954160 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954163 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954166 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954169 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954172 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954178 2582 flags.go:64] FLAG: --cgroup-root="" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954181 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954184 2582 flags.go:64] FLAG: --client-ca-file="" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954187 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954190 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954195 2582 flags.go:64] FLAG: --cluster-domain="" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954198 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954202 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 27 16:56:33.959012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954205 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954208 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954213 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954216 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954219 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954223 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954226 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954229 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954232 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954235 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954238 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954242 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954245 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954248 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954251 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954254 2582 flags.go:64] FLAG: --enable-server="true" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954258 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954263 2582 flags.go:64] FLAG: --event-burst="100" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954266 2582 flags.go:64] FLAG: --event-qps="50" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954269 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954272 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954275 2582 flags.go:64] FLAG: --eviction-hard="" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954279 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954282 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954286 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 27 16:56:33.959724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954289 2582 flags.go:64] FLAG: --eviction-soft="" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954292 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954295 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954298 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954301 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954304 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954307 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954310 2582 flags.go:64] FLAG: --feature-gates="" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954314 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954317 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954320 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954323 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954326 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954329 2582 flags.go:64] FLAG: --help="false" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954332 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-135-54.ec2.internal" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954335 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954338 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954341 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954344 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954347 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954350 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954353 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954356 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 27 16:56:33.960327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954359 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954363 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954366 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954369 2582 flags.go:64] FLAG: --kube-reserved="" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954372 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954387 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954390 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954393 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954397 2582 flags.go:64] FLAG: --lock-file="" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954400 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954403 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954406 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954412 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954415 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954417 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954420 2582 flags.go:64] FLAG: --logging-format="text" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954423 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954427 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954429 2582 flags.go:64] FLAG: --manifest-url="" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954432 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954437 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954440 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954444 2582 flags.go:64] FLAG: --max-pods="110" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954447 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954449 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 27 16:56:33.960945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954453 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954456 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954459 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954461 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954464 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954471 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954475 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954478 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954481 2582 flags.go:64] FLAG: --pod-cidr="" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954484 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954490 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954493 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954496 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954499 2582 flags.go:64] FLAG: --port="10250" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954502 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954505 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01b09b8e496731084" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954510 2582 flags.go:64] FLAG: --qos-reserved="" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954513 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954516 2582 flags.go:64] FLAG: --register-node="true" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954519 2582 flags.go:64] FLAG: --register-with-taints="" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954523 2582 flags.go:64] FLAG: --registry-burst="10" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954525 2582 flags.go:64] FLAG: --registry-qps="5" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954528 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954531 2582 flags.go:64] FLAG: --reserved-memory="" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954535 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 27 16:56:33.961602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954539 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954542 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954545 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954548 2582 flags.go:64] FLAG: --runonce="false" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954551 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954553 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954557 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954560 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954563 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954566 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954569 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954572 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954575 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954578 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954583 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954585 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954589 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954592 2582 flags.go:64] FLAG: --system-cgroups="" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954595 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954600 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954603 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954606 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954610 2582 flags.go:64] FLAG: --tls-min-version="" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954614 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954617 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 27 16:56:33.962261 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954619 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954622 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954625 2582 flags.go:64] FLAG: --v="2" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954629 2582 flags.go:64] FLAG: --version="false" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954633 2582 flags.go:64] FLAG: --vmodule="" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954638 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.954641 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954756 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954760 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954764 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954767 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954770 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954773 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954776 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954779 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954783 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954786 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954789 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954792 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954794 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 16:56:33.962888 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954797 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954802 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954805 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954807 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954810 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954813 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954816 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954818 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954821 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954824 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954827 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954830 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954832 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954835 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954838 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954840 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954843 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954845 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954848 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 16:56:33.963396 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954850 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954853 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954855 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954858 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954861 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954863 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954866 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954868 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954871 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954873 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954876 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954878 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954881 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954883 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954887 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954890 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954893 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954895 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954898 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954901 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 16:56:33.963859 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954904 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954906 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954909 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954913 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954915 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954918 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954920 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954923 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954926 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954928 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954931 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954933 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954936 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954938 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954941 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954943 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954946 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954948 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954951 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954953 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 16:56:33.964354 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954956 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954960 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954963 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954966 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954969 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954972 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954976 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954979 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954981 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954984 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954986 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954989 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954992 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954995 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.954998 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955002 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955005 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955008 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955010 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955013 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 16:56:33.964968 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955016 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955019 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955022 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955024 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955027 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955029 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.955032 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.955041 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.961797 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.961818 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.961839 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.961846 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961906 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961911 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961914 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 27 16:56:33.965468 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961917 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961921 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961924 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961927 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961929 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961932 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961935 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961937 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961940 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961943 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961945 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961948 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961950 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961953 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961955 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961958 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961960 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961964 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961966 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961968 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 16:56:33.965832 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961971 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961974 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961976 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961979 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961982 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961984 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961989 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961991 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961994 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961997 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.961999 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962002 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962004 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962007 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962009 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962012 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962015 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962017 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962020 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962023 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 16:56:33.966311 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962025 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962028 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962030 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962033 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962036 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962038 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962041 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962044 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962046 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962049 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962052 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962054 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962057 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962060 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962062 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962065 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962067 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962070 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962074 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962077 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 16:56:33.966812 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962080 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962082 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962085 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962087 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962090 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962093 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962095 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962098 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962100 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962103 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962106 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962108 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962113 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962116 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962119 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962122 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962125 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962129 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962132 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 16:56:33.967310 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962136 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962138 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962141 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962144 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962146 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962148 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962151 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962154 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962156 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962159 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962162 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962166 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962170 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962174 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962176 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962178 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 16:56:33.967774 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962181 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.962186 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962299 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962304 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962307 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962310 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962312 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962315 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962318 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962320 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962323 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962325 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962328 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962330 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962333 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 16:56:33.968170 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962335 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962338 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962340 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962343 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962345 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962348 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962350 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962353 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962355 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962358 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962361 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962364 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962368 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962370 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962373 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962389 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962392 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962395 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962398 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962401 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 16:56:33.968589 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962403 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962406 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962410 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962413 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962416 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962418 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962421 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962424 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962426 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962429 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962431 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962434 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962437 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962439 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962442 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962444 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962447 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962449 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962452 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 16:56:33.969056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962454 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962456 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962459 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962461 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962464 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962467 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962469 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962473 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962477 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962480 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962484 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962487 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962490 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962492 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962494 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962497 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962501 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962504 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962506 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 16:56:33.969524 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962509 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962511 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962514 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962516 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962519 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962521 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962524 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962527 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962529 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962532 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962534 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962537 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962539 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962542 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962544 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962547 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962549 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962552 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962555 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962558 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 16:56:33.969993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962560 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962563 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962565 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962569 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962572 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962574 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962577 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:33.962579 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.962584 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.962709 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.964747 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.965915 2582 server.go:1019] "Starting client certificate rotation" Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.966017 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 16:56:33.970511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.966076 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 16:56:33.970828 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.968523 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 16:56:33.970828 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.969887 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 16:56:33.981883 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.981863 2582 log.go:25] "Validated CRI v1 runtime API" Jul 27 16:56:33.984166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.984150 2582 log.go:25] "Validated CRI v1 image API" Jul 27 16:56:33.984971 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.984958 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 27 16:56:33.987191 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.987162 2582 fs.go:135] Filesystem UUIDs: map[13d24028-5bcc-4839-97af-9eef7cdf6856:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 92c64f0b-3230-42b2-a2b3-8d2ac25a74e4:/dev/nvme0n1p3] Jul 27 16:56:33.987264 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.987189 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 27 16:56:33.992472 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.992328 2582 manager.go:216] Machine: {Timestamp:2026-07-27 16:56:33.991171247 +0000 UTC m=+0.275330522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100057 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fbc69b92024a97e592c0cfc48bc7d SystemUUID:ec2fbc69-b920-24a9-7e59-2c0cfc48bc7d BootID:8e2de628-4e47-477a-99a0-8de9f63d8107 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3a:44:4d:c2:35 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3a:44:4d:c2:35 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:e5:42:f0:f7:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 27 16:56:33.992472 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.992458 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 27 16:56:33.992666 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.992628 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 27 16:56:33.993594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.993564 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 27 16:56:33.993734 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.993598 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 27 16:56:33.994370 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.994361 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 27 16:56:33.994415 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.994373 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 27 16:56:33.994415 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.994395 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 16:56:33.994415 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.994410 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 16:56:33.995125 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.995115 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 27 16:56:33.996640 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996627 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 27 16:56:33.996765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996756 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 27 16:56:33.996801 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996792 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 27 16:56:33.996829 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996802 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 27 16:56:33.996829 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996814 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 27 16:56:33.996829 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996823 2582 kubelet.go:398] "Adding apiserver pod source" Jul 27 16:56:33.996912 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.996831 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 27 16:56:33.997294 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.997280 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 16:56:33.997344 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.997308 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 16:56:33.998025 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.998002 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 16:56:33.999123 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:33.999101 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 27 16:56:34.000560 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.000546 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 27 16:56:34.000604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.000577 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 27 16:56:34.002082 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002071 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 27 16:56:34.002126 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002096 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 27 16:56:34.002126 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002103 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 27 16:56:34.002126 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002109 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 27 16:56:34.002126 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002116 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 27 16:56:34.002126 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002123 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 27 16:56:34.002253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002130 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 27 16:56:34.002253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002135 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 27 16:56:34.002253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002143 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 27 16:56:34.002253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002148 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 27 16:56:34.002253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002156 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 27 16:56:34.002253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.002161 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 27 16:56:34.003364 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.003354 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 27 16:56:34.003420 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.003367 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 27 16:56:34.006507 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.006493 2582 server.go:1268] "Started kubelet" Jul 27 16:56:34.006612 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.006548 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 27 16:56:34.006612 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.006559 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 27 16:56:34.006713 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.006633 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 27 16:56:34.007486 ip-10-0-135-54 systemd[1]: Started Kubernetes Kubelet. Jul 27 16:56:34.009860 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.009841 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 27 16:56:34.011935 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.010944 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 27 16:56:34.013506 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.013485 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 27 16:56:34.015763 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.015742 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 27 16:56:34.015826 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.015742 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 27 16:56:34.015826 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.015792 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 27 16:56:34.016206 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.016192 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 27 16:56:34.016251 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.016203 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 27 16:56:34.016489 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.015685 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-54.ec2.internal.18c633ecb2f1c2ac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-54.ec2.internal,UID:ip-10-0-135-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-54.ec2.internal,},FirstTimestamp:2026-07-27 16:56:34.006467244 +0000 UTC m=+0.290626518,LastTimestamp:2026-07-27 16:56:34.006467244 +0000 UTC m=+0.290626518,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-54.ec2.internal,}" Jul 27 16:56:34.016618 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.016518 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.016869 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.016847 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 27 16:56:34.016966 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.016850 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 27 16:56:34.016966 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.016885 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 27 16:56:34.017066 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.016999 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 27 16:56:34.017066 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017003 2582 factory.go:55] Registering systemd factory Jul 27 16:56:34.017066 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017019 2582 factory.go:223] Registration of the systemd container factory successfully Jul 27 16:56:34.017066 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017008 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 27 16:56:34.017242 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017209 2582 factory.go:153] Registering CRI-O factory Jul 27 16:56:34.017242 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017226 2582 factory.go:223] Registration of the crio container factory successfully Jul 27 16:56:34.017334 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017275 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 27 16:56:34.017334 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017298 2582 factory.go:103] Registering Raw factory Jul 27 16:56:34.017334 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017313 2582 manager.go:1196] Started watching for new ooms in manager Jul 27 16:56:34.017959 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.017860 2582 manager.go:349] Starting recovery of all containers Jul 27 16:56:34.023587 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.023547 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 27 16:56:34.023703 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.023654 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 27 16:56:34.028699 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.028685 2582 manager.go:354] Recovery completed Jul 27 16:56:34.030986 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.030966 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zt9rj" Jul 27 16:56:34.033345 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.033332 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.035309 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.035294 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.035371 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.035321 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.035371 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.035333 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.035936 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.035920 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 27 16:56:34.035936 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.035933 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 27 16:56:34.036040 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.035948 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 27 16:56:34.036040 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.036008 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.037103 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.037083 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.037155 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.037115 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.037155 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.037126 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.038403 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.038359 2582 policy_none.go:49] "None policy: Start" Jul 27 16:56:34.038494 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.038409 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 27 16:56:34.038494 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.038425 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 27 16:56:34.039472 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.038482 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-54.ec2.internal.18c633ecb4a9d4ee default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-54.ec2.internal,UID:ip-10-0-135-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-54.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-54.ec2.internal,},FirstTimestamp:2026-07-27 16:56:34.035307758 +0000 UTC m=+0.319467035,LastTimestamp:2026-07-27 16:56:34.035307758 +0000 UTC m=+0.319467035,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-54.ec2.internal,}" Jul 27 16:56:34.040270 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.040253 2582 policy_none.go:47] "Start" Jul 27 16:56:34.040712 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.040692 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zt9rj" Jul 27 16:56:34.082169 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.081995 2582 manager.go:337] "Starting Device Plugin manager" Jul 27 16:56:34.082276 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.082179 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 27 16:56:34.082276 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.082191 2582 server.go:87] "Starting device plugin registration server" Jul 27 16:56:34.082462 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.082449 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 27 16:56:34.082513 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.082464 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 27 16:56:34.082567 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.082546 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 27 16:56:34.082633 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.082622 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 27 16:56:34.082672 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.082634 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 27 16:56:34.083229 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.083201 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 27 16:56:34.083307 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.083256 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.182737 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.182693 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.183764 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.183748 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.183841 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.183779 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.183841 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.183790 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.183841 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.183816 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.190561 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.190535 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 27 16:56:34.191742 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.191727 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 27 16:56:34.191800 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.191765 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 27 16:56:34.191800 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.191787 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 27 16:56:34.191877 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.191819 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 27 16:56:34.192286 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.192270 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.192409 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.192297 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-54.ec2.internal\": node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.193255 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.193237 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 16:56:34.241616 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.241507 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.292645 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.292608 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal"] Jul 27 16:56:34.292746 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.292693 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.293746 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.293729 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.293847 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.293758 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.293847 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.293768 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.295010 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.294997 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.295130 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295116 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.295202 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295145 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.295769 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295752 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.295852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295780 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.295852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295796 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.295852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295834 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.295852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295853 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.296039 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.295864 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.296880 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.296864 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.296957 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.296890 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 16:56:34.297568 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.297551 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientMemory" Jul 27 16:56:34.297642 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.297578 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 16:56:34.297642 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.297588 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeHasSufficientPID" Jul 27 16:56:34.318951 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.318924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.318951 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.318951 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.319101 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.318971 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e5ee8caab48bd3dff1dc3efc23e7297-config\") pod \"kube-apiserver-proxy-ip-10-0-135-54.ec2.internal\" (UID: \"1e5ee8caab48bd3dff1dc3efc23e7297\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.324193 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.324174 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-54.ec2.internal\" not found" node="ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.328723 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.328707 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-54.ec2.internal\" not found" node="ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.341776 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.341755 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.420130 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.420106 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.420130 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.420133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.420303 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.420150 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e5ee8caab48bd3dff1dc3efc23e7297-config\") pod \"kube-apiserver-proxy-ip-10-0-135-54.ec2.internal\" (UID: \"1e5ee8caab48bd3dff1dc3efc23e7297\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.420303 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.420191 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1e5ee8caab48bd3dff1dc3efc23e7297-config\") pod \"kube-apiserver-proxy-ip-10-0-135-54.ec2.internal\" (UID: \"1e5ee8caab48bd3dff1dc3efc23e7297\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.420303 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.420210 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.420303 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.420229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ddbfdf1620eb095b72601b0c1530585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal\" (UID: \"2ddbfdf1620eb095b72601b0c1530585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.442826 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.442801 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.544211 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.544116 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.627471 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.627425 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.631317 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.631174 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 27 16:56:34.644595 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.644570 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.745808 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.745760 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.847583 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.847549 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.948622 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:34.948589 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:34.966049 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.966016 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 27 16:56:34.966204 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:34.966181 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 16:56:35.044148 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.044106 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 16:51:34 +0000 UTC" deadline="2027-12-27 22:12:28.808635133 +0000 UTC" Jul 27 16:56:35.044148 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.044140 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12437h15m53.764499818s" Jul 27 16:56:35.049520 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:35.049499 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:35.061577 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.061554 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 16:56:35.150096 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:35.150020 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:35.251164 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:35.251134 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:35.300257 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:35.300210 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5ee8caab48bd3dff1dc3efc23e7297.slice/crio-534a3ec2a48d0fad6146e6da5a0be5d7ae0f3e80f680d3136ae41b05b106837c WatchSource:0}: Error finding container 534a3ec2a48d0fad6146e6da5a0be5d7ae0f3e80f680d3136ae41b05b106837c: Status 404 returned error can't find the container with id 534a3ec2a48d0fad6146e6da5a0be5d7ae0f3e80f680d3136ae41b05b106837c Jul 27 16:56:35.300516 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:35.300495 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ddbfdf1620eb095b72601b0c1530585.slice/crio-a037d29066b8002a6892522d8280aa3d990581cdf8761a518090ff9477d481ec WatchSource:0}: Error finding container a037d29066b8002a6892522d8280aa3d990581cdf8761a518090ff9477d481ec: Status 404 returned error can't find the container with id a037d29066b8002a6892522d8280aa3d990581cdf8761a518090ff9477d481ec Jul 27 16:56:35.307077 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.307060 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 16:56:35.353014 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:35.352967 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-54.ec2.internal\" not found" Jul 27 16:56:35.394316 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.394292 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 16:56:35.414503 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.414439 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 16:56:35.416595 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.416574 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" Jul 27 16:56:35.430394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.430360 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 16:56:35.431356 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.431341 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" Jul 27 16:56:35.441368 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.441352 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 16:56:35.774354 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.774267 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 16:56:35.999802 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:35.999535 2582 apiserver.go:52] "Watching apiserver" Jul 27 16:56:36.009908 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.009875 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 27 16:56:36.010573 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.010541 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-nk6nm","kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h","openshift-cluster-node-tuning-operator/tuned-22rdd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal","openshift-multus/multus-additional-cni-plugins-s9pk6","openshift-multus/multus-k46vl","openshift-multus/network-metrics-daemon-bx5ct","kube-system/global-pull-secret-syncer-kb9bl","openshift-dns/node-resolver-vhnf5","openshift-image-registry/node-ca-v5gbj","openshift-network-diagnostics/network-check-target-rhgrq","openshift-network-operator/iptables-alerter-qrq8b","openshift-ovn-kubernetes/ovnkube-node-lzcb8"] Jul 27 16:56:36.013446 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.013419 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.015981 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.015956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.016571 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.016521 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 27 16:56:36.016663 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.016614 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.016981 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.016952 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 27 16:56:36.017166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.017149 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 27 16:56:36.017415 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.017364 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.017415 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.017403 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2z82r\"" Jul 27 16:56:36.018974 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.018942 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 27 16:56:36.019184 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.019167 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 27 16:56:36.019742 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.019722 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-br6tc\"" Jul 27 16:56:36.020021 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.020004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:36.020146 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.020077 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:36.021293 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.021274 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.023543 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.023523 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.024582 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.024530 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.024782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.024764 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5hxjn\"" Jul 27 16:56:36.025122 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.025102 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.026212 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.026188 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.026311 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.026253 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.026374 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.026331 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:36.026559 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.026533 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.026711 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.026678 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 27 16:56:36.027028 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.026891 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dlz5d\"" Jul 27 16:56:36.028744 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.028721 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.030805 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.030771 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 16:56:36.031186 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031159 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-cni-multus\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031294 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031222 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-os-release\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.031294 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031267 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-etc-kubernetes\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031434 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031293 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.031434 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-cni-binary-copy\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.031434 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.031434 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81f91d24-0107-4993-bddc-4d8a906f11ff-cni-binary-copy\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031434 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-cnibin\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031439 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-netns\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031494 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-cnibin\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031569 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-daemon-config\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031595 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-hostroot\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031638 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.031698 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkrxm\" (UniqueName: \"kubernetes.io/projected/e0d78532-02c9-4292-aa11-7f235a427af1-kube-api-access-dkrxm\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.032045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031702 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-cni-bin\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031729 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-system-cni-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031772 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-cni-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.031795 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-os-release\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032278 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.032067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-socket-dir-parent\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032278 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.032115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-k8s-cni-cncf-io\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032278 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.032149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-kubelet\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032278 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.032182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-conf-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.032278 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.032210 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-multus-certs\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.033317 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.033488 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn94r\" (UniqueName: \"kubernetes.io/projected/81f91d24-0107-4993-bddc-4d8a906f11ff-kube-api-access-mn94r\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.033593 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-system-cni-dir\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.033680 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.034300 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.035032 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.035041 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-55lfh\"" Jul 27 16:56:36.037292 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.035459 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.038169 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.037681 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040073 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9vw9d\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040309 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040390 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040429 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040578 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040693 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4bpn6\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040790 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040818 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040876 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.040981 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.041069 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:36.042394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.042240 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zqj6x\"" Jul 27 16:56:36.043577 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.043270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.046150 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.046127 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 27 16:56:36.046810 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.046791 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 27 16:56:36.047190 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.047171 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 27 16:56:36.047892 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.047839 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 27 16:56:36.048112 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.048093 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 27 16:56:36.048336 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.048299 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dsnbw\"" Jul 27 16:56:36.048515 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.048496 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 27 16:56:36.050992 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.050959 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fwss" Jul 27 16:56:36.061545 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.061515 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fwss" Jul 27 16:56:36.118303 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.118151 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 27 16:56:36.134459 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-cnibin\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.134604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134460 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntz9j\" (UniqueName: \"kubernetes.io/projected/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-kube-api-access-ntz9j\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.134604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-run\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.134604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.134604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134539 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-node-log\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.134604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-cnibin\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.134604 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-cni-bin\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-daemon-config\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-dbus\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134761 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-hostroot\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-serviceca\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134852 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9b633ee-da5c-4a20-98d1-ad5688caf314-hosts-file\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-registration-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-slash\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134915 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-etc-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.134937 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rwv8\" (UniqueName: \"kubernetes.io/projected/76fc5ab0-111f-4f67-96d6-a030f5c61865-kube-api-access-5rwv8\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134952 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkrxm\" (UniqueName: \"kubernetes.io/projected/e0d78532-02c9-4292-aa11-7f235a427af1-kube-api-access-dkrxm\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.134996 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-cni-bin\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135034 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-system-cni-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-cni-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135081 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-os-release\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135106 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76fc5ab0-111f-4f67-96d6-a030f5c61865-host-slash\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135132 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-multus-certs\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135179 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysctl-d\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135191 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-daemon-config\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135243 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-run-ovn-kubernetes\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.135322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-ovnkube-script-lib\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135334 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-lib-modules\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-kubelet-config\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-cni-multus\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135511 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-hostroot\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135521 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-os-release\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135561 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76fc5ab0-111f-4f67-96d6-a030f5c61865-iptables-alerter-script\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-device-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135640 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-cni-bin\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135643 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-ovnkube-config\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135685 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-env-overrides\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135714 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-etc-kubernetes\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135800 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81f91d24-0107-4993-bddc-4d8a906f11ff-cni-binary-copy\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hp7\" (UniqueName: \"kubernetes.io/projected/d9b633ee-da5c-4a20-98d1-ad5688caf314-kube-api-access-78hp7\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.135850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-os-release\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-var-lib-kubelet\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clkn5\" (UniqueName: \"kubernetes.io/projected/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-kube-api-access-clkn5\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135912 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-os-release\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135924 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-cnibin\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135954 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-system-cni-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135973 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-cnibin\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136007 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-etc-kubernetes\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136076 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-multus-certs\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.135793 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-cni-multus\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136160 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-cni-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136213 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-netns\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a607406-2852-40c5-9633-b4c07b3205f0-tmp\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136303 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-kubelet\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-systemd-units\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136352 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-run-netns\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136400 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-log-socket\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.136765 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136471 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a607406-2852-40c5-9633-b4c07b3205f0-etc-tuned\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136512 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-socket-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136542 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-etc-selinux\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136566 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-systemd\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-cni-netd\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136615 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8klh\" (UniqueName: \"kubernetes.io/projected/0df410a3-75d4-4715-a942-be270f0d0117-kube-api-access-n8klh\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bd31b56-43d8-4f43-9a3e-28ffbab38410-konnectivity-ca\") pod \"konnectivity-agent-nk6nm\" (UID: \"5bd31b56-43d8-4f43-9a3e-28ffbab38410\") " pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136660 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136681 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136710 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0df410a3-75d4-4715-a942-be270f0d0117-ovn-node-metrics-cert\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136752 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bd31b56-43d8-4f43-9a3e-28ffbab38410-agent-certs\") pod \"konnectivity-agent-nk6nm\" (UID: \"5bd31b56-43d8-4f43-9a3e-28ffbab38410\") " pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136831 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81f91d24-0107-4993-bddc-4d8a906f11ff-cni-binary-copy\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136861 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136901 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-netns\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.136888 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-socket-dir-parent\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-k8s-cni-cncf-io\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn94r\" (UniqueName: \"kubernetes.io/projected/81f91d24-0107-4993-bddc-4d8a906f11ff-kube-api-access-mn94r\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.137715 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137086 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-socket-dir-parent\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137114 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-system-cni-dir\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137192 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-system-cni-dir\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137246 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-run-k8s-cni-cncf-io\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137288 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0d78532-02c9-4292-aa11-7f235a427af1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-sys-fs\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137372 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp7hw\" (UniqueName: \"kubernetes.io/projected/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-kube-api-access-rp7hw\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-sys\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137504 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-kubelet\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137555 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-conf-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysconfig\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysctl-conf\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137623 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-multus-conf-dir\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137616 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/81f91d24-0107-4993-bddc-4d8a906f11ff-host-var-lib-kubelet\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-systemd\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-modprobe-d\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-kubernetes\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.138594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137803 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-ovn\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137832 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9b633ee-da5c-4a20-98d1-ad5688caf314-tmp-dir\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-host\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k68ng\" (UniqueName: \"kubernetes.io/projected/6a607406-2852-40c5-9633-b4c07b3205f0-kube-api-access-k68ng\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137914 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-var-lib-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.137949 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-cni-binary-copy\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.138019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.138058 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-host\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.138552 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-cni-binary-copy\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.139442 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.138812 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e0d78532-02c9-4292-aa11-7f235a427af1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.142953 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.142921 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 27 16:56:36.144234 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.143871 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:56:36.144234 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.143896 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:56:36.144234 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.143909 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:36.144234 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.143989 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:36.643958956 +0000 UTC m=+2.928118227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:36.147839 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.147816 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkrxm\" (UniqueName: \"kubernetes.io/projected/e0d78532-02c9-4292-aa11-7f235a427af1-kube-api-access-dkrxm\") pod \"multus-additional-cni-plugins-s9pk6\" (UID: \"e0d78532-02c9-4292-aa11-7f235a427af1\") " pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.148012 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.147989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn94r\" (UniqueName: \"kubernetes.io/projected/81f91d24-0107-4993-bddc-4d8a906f11ff-kube-api-access-mn94r\") pod \"multus-k46vl\" (UID: \"81f91d24-0107-4993-bddc-4d8a906f11ff\") " pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.198691 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.198632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" event={"ID":"1e5ee8caab48bd3dff1dc3efc23e7297","Type":"ContainerStarted","Data":"534a3ec2a48d0fad6146e6da5a0be5d7ae0f3e80f680d3136ae41b05b106837c"} Jul 27 16:56:36.200271 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.200239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" event={"ID":"2ddbfdf1620eb095b72601b0c1530585","Type":"ContainerStarted","Data":"a037d29066b8002a6892522d8280aa3d990581cdf8761a518090ff9477d481ec"} Jul 27 16:56:36.239456 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239413 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-host\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239466 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntz9j\" (UniqueName: \"kubernetes.io/projected/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-kube-api-access-ntz9j\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239492 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-run\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239538 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-node-log\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-cni-bin\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239582 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-dbus\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-serviceca\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.239639 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9b633ee-da5c-4a20-98d1-ad5688caf314-hosts-file\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239648 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-registration-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239672 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-slash\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239697 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-etc-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239775 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-slash\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239781 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-cni-bin\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239796 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d9b633ee-da5c-4a20-98d1-ad5688caf314-hosts-file\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rwv8\" (UniqueName: \"kubernetes.io/projected/76fc5ab0-111f-4f67-96d6-a030f5c61865-kube-api-access-5rwv8\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239852 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-dbus\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239858 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-etc-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239880 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76fc5ab0-111f-4f67-96d6-a030f5c61865-host-slash\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239908 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysctl-d\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239936 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.239955 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239961 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-run-ovn-kubernetes\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.239993 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-ovnkube-script-lib\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-lib-modules\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.240045 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.240040 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:36.740017404 +0000 UTC m=+3.024176696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240073 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-kubelet-config\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76fc5ab0-111f-4f67-96d6-a030f5c61865-iptables-alerter-script\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240135 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-device-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240170 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-lib-modules\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-ovnkube-config\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-env-overrides\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-node-log\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240302 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240263 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240311 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-serviceca\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hp7\" (UniqueName: \"kubernetes.io/projected/d9b633ee-da5c-4a20-98d1-ad5688caf314-kube-api-access-78hp7\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240362 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-run\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-var-lib-kubelet\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-host\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240402 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clkn5\" (UniqueName: \"kubernetes.io/projected/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-kube-api-access-clkn5\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240445 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a607406-2852-40c5-9633-b4c07b3205f0-tmp\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240488 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-kubelet\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.240914 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240515 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-systemd-units\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240539 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-run-netns\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240566 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-log-socket\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a607406-2852-40c5-9633-b4c07b3205f0-etc-tuned\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240617 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysctl-d\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76fc5ab0-111f-4f67-96d6-a030f5c61865-host-slash\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240618 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-socket-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.240699 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.240754 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:36.740735163 +0000 UTC m=+3.024894432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240751 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-run-ovn-kubernetes\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240762 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-kubelet-config\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240752 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-var-lib-kubelet\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240785 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-systemd-units\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240844 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-device-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-socket-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241000 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-etc-selinux\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-log-socket\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.241853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-etc-selinux\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241080 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-systemd\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-cni-netd\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241135 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8klh\" (UniqueName: \"kubernetes.io/projected/0df410a3-75d4-4715-a942-be270f0d0117-kube-api-access-n8klh\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bd31b56-43d8-4f43-9a3e-28ffbab38410-konnectivity-ca\") pod \"konnectivity-agent-nk6nm\" (UID: \"5bd31b56-43d8-4f43-9a3e-28ffbab38410\") " pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241218 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0df410a3-75d4-4715-a942-be270f0d0117-ovn-node-metrics-cert\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241243 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bd31b56-43d8-4f43-9a3e-28ffbab38410-agent-certs\") pod \"konnectivity-agent-nk6nm\" (UID: \"5bd31b56-43d8-4f43-9a3e-28ffbab38410\") " pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241261 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-systemd\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241276 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-sys-fs\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241264 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-registration-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.240999 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-kubelet\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp7hw\" (UniqueName: \"kubernetes.io/projected/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-kube-api-access-rp7hw\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241332 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-sys\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241341 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241361 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysconfig\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241388 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysctl-conf\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.242732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241409 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-cni-netd\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241415 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-systemd\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241414 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76fc5ab0-111f-4f67-96d6-a030f5c61865-iptables-alerter-script\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-modprobe-d\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241505 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-kubernetes\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241538 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-modprobe-d\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241565 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-ovn\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241590 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9b633ee-da5c-4a20-98d1-ad5688caf314-tmp-dir\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241611 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-sys\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241616 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-host\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241676 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-host\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241728 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-systemd\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241777 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysctl-conf\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241854 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-run-ovn\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241998 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-host-run-netns\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242078 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-sys-fs\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.243591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242087 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bd31b56-43d8-4f43-9a3e-28ffbab38410-konnectivity-ca\") pod \"konnectivity-agent-nk6nm\" (UID: \"5bd31b56-43d8-4f43-9a3e-28ffbab38410\") " pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-kubernetes\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k68ng\" (UniqueName: \"kubernetes.io/projected/6a607406-2852-40c5-9633-b4c07b3205f0-kube-api-access-k68ng\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242198 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-var-lib-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242212 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9b633ee-da5c-4a20-98d1-ad5688caf314-tmp-dir\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.241675 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a607406-2852-40c5-9633-b4c07b3205f0-etc-sysconfig\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242273 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0df410a3-75d4-4715-a942-be270f0d0117-var-lib-openvswitch\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242333 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-ovnkube-config\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.242811 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-ovnkube-script-lib\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.244523 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.243319 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0df410a3-75d4-4715-a942-be270f0d0117-env-overrides\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.246510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.246486 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bd31b56-43d8-4f43-9a3e-28ffbab38410-agent-certs\") pod \"konnectivity-agent-nk6nm\" (UID: \"5bd31b56-43d8-4f43-9a3e-28ffbab38410\") " pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.246654 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.246632 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a607406-2852-40c5-9633-b4c07b3205f0-tmp\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.246828 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.246804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0df410a3-75d4-4715-a942-be270f0d0117-ovn-node-metrics-cert\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.247497 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.247408 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a607406-2852-40c5-9633-b4c07b3205f0-etc-tuned\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.258532 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.258305 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hp7\" (UniqueName: \"kubernetes.io/projected/d9b633ee-da5c-4a20-98d1-ad5688caf314-kube-api-access-78hp7\") pod \"node-resolver-vhnf5\" (UID: \"d9b633ee-da5c-4a20-98d1-ad5688caf314\") " pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.260400 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.260346 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntz9j\" (UniqueName: \"kubernetes.io/projected/cd1e1b48-2cc3-4e96-9bec-a23842e66afb-kube-api-access-ntz9j\") pod \"node-ca-v5gbj\" (UID: \"cd1e1b48-2cc3-4e96-9bec-a23842e66afb\") " pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.260400 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.260363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k68ng\" (UniqueName: \"kubernetes.io/projected/6a607406-2852-40c5-9633-b4c07b3205f0-kube-api-access-k68ng\") pod \"tuned-22rdd\" (UID: \"6a607406-2852-40c5-9633-b4c07b3205f0\") " pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.260400 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.260363 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rwv8\" (UniqueName: \"kubernetes.io/projected/76fc5ab0-111f-4f67-96d6-a030f5c61865-kube-api-access-5rwv8\") pod \"iptables-alerter-qrq8b\" (UID: \"76fc5ab0-111f-4f67-96d6-a030f5c61865\") " pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.261254 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.261206 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp7hw\" (UniqueName: \"kubernetes.io/projected/a3c2d2cc-2246-4e1d-b800-2eea9f7be449-kube-api-access-rp7hw\") pod \"aws-ebs-csi-driver-node-vzs2h\" (UID: \"a3c2d2cc-2246-4e1d-b800-2eea9f7be449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.262036 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.261981 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8klh\" (UniqueName: \"kubernetes.io/projected/0df410a3-75d4-4715-a942-be270f0d0117-kube-api-access-n8klh\") pod \"ovnkube-node-lzcb8\" (UID: \"0df410a3-75d4-4715-a942-be270f0d0117\") " pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.262456 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.262432 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clkn5\" (UniqueName: \"kubernetes.io/projected/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-kube-api-access-clkn5\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.331703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.331601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k46vl" Jul 27 16:56:36.350087 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.350059 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" Jul 27 16:56:36.358269 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.358229 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-22rdd" Jul 27 16:56:36.366465 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.366427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5gbj" Jul 27 16:56:36.375712 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.375687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qrq8b" Jul 27 16:56:36.383512 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.383490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhnf5" Jul 27 16:56:36.394363 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.394342 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:56:36.400209 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.400190 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" Jul 27 16:56:36.408171 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.408150 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:56:36.645194 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.645153 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:36.645420 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.645375 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:56:36.645560 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.645424 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:56:36.645560 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.645438 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:36.645560 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.645512 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:37.645486719 +0000 UTC m=+3.929645988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:36.746741 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.746698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:36.746940 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:36.746802 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:36.747101 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.747015 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:36.747178 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.747104 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:37.747083548 +0000 UTC m=+4.031242831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:36.747670 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.747648 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:36.747767 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:36.747714 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:37.747689477 +0000 UTC m=+4.031848756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:37.065355 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:37.065224 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 16:51:36 +0000 UTC" deadline="2028-04-29 16:57:23.37377589 +0000 UTC" Jul 27 16:56:37.065355 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:37.065298 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15408h0m46.308481827s" Jul 27 16:56:37.192372 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:37.192345 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:37.192540 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.192455 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:37.286890 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.286854 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fc5ab0_111f_4f67_96d6_a030f5c61865.slice/crio-99a61c06be911652af2116acfecdf98e530ec7ba9a58193fca02c4d66a76a207 WatchSource:0}: Error finding container 99a61c06be911652af2116acfecdf98e530ec7ba9a58193fca02c4d66a76a207: Status 404 returned error can't find the container with id 99a61c06be911652af2116acfecdf98e530ec7ba9a58193fca02c4d66a76a207 Jul 27 16:56:37.291332 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.291233 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a607406_2852_40c5_9633_b4c07b3205f0.slice/crio-d81b26f55d229025cbd2c0edeee2a88fd99bbbe3c5e77a8a7ef3dee6f0d45556 WatchSource:0}: Error finding container d81b26f55d229025cbd2c0edeee2a88fd99bbbe3c5e77a8a7ef3dee6f0d45556: Status 404 returned error can't find the container with id d81b26f55d229025cbd2c0edeee2a88fd99bbbe3c5e77a8a7ef3dee6f0d45556 Jul 27 16:56:37.292874 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.292854 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df410a3_75d4_4715_a942_be270f0d0117.slice/crio-1f112c74441d00d64a62062959c76c85078f4d72b1eb43a1b828465c91683574 WatchSource:0}: Error finding container 1f112c74441d00d64a62062959c76c85078f4d72b1eb43a1b828465c91683574: Status 404 returned error can't find the container with id 1f112c74441d00d64a62062959c76c85078f4d72b1eb43a1b828465c91683574 Jul 27 16:56:37.293964 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.293939 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f91d24_0107_4993_bddc_4d8a906f11ff.slice/crio-dfead470de8881cb7353d8aed0bd189542b60cb524ee129593a4094e67c04864 WatchSource:0}: Error finding container dfead470de8881cb7353d8aed0bd189542b60cb524ee129593a4094e67c04864: Status 404 returned error can't find the container with id dfead470de8881cb7353d8aed0bd189542b60cb524ee129593a4094e67c04864 Jul 27 16:56:37.295623 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.295585 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd31b56_43d8_4f43_9a3e_28ffbab38410.slice/crio-55cb830ed987f0bce8ddce27a2e4e1a43f3ff25d0ba044e563cc48decf65985c WatchSource:0}: Error finding container 55cb830ed987f0bce8ddce27a2e4e1a43f3ff25d0ba044e563cc48decf65985c: Status 404 returned error can't find the container with id 55cb830ed987f0bce8ddce27a2e4e1a43f3ff25d0ba044e563cc48decf65985c Jul 27 16:56:37.297358 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.297228 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b633ee_da5c_4a20_98d1_ad5688caf314.slice/crio-1179d06d533d13765b1d27a725419a08088777518270ae57fd3a08bcb81cd54e WatchSource:0}: Error finding container 1179d06d533d13765b1d27a725419a08088777518270ae57fd3a08bcb81cd54e: Status 404 returned error can't find the container with id 1179d06d533d13765b1d27a725419a08088777518270ae57fd3a08bcb81cd54e Jul 27 16:56:37.298330 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:56:37.298027 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c2d2cc_2246_4e1d_b800_2eea9f7be449.slice/crio-038ad826f37a008f7efba48d1780a077e9823a96fc07cac1b5fa4fcc30930d10 WatchSource:0}: Error finding container 038ad826f37a008f7efba48d1780a077e9823a96fc07cac1b5fa4fcc30930d10: Status 404 returned error can't find the container with id 038ad826f37a008f7efba48d1780a077e9823a96fc07cac1b5fa4fcc30930d10 Jul 27 16:56:37.655733 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:37.655524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:37.655890 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.655706 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:56:37.655890 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.655774 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:56:37.655890 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.655786 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:37.655890 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.655841 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:39.655827261 +0000 UTC m=+5.939986523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:37.756999 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:37.756958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:37.757179 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:37.757020 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:37.757179 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.757110 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:37.757179 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.757131 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:37.757287 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.757180 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:39.757162015 +0000 UTC m=+6.041321299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:37.757287 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:37.757197 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:39.757189498 +0000 UTC m=+6.041348760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:38.065953 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.065856 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 16:51:36 +0000 UTC" deadline="2028-04-22 07:46:45.725000354 +0000 UTC" Jul 27 16:56:38.065953 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.065901 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15230h50m7.659104211s" Jul 27 16:56:38.192835 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.192404 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:38.192835 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.192454 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:38.192835 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:38.192562 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:38.192835 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:38.192714 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:38.204707 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.204674 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-22rdd" event={"ID":"6a607406-2852-40c5-9633-b4c07b3205f0","Type":"ContainerStarted","Data":"d81b26f55d229025cbd2c0edeee2a88fd99bbbe3c5e77a8a7ef3dee6f0d45556"} Jul 27 16:56:38.207973 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.207559 2582 generic.go:356] "Generic (PLEG): container finished" podID="2ddbfdf1620eb095b72601b0c1530585" containerID="ca299be2b7fc5627c40bfd4724a1b5ba96271089d110392c90c25b2d900ecdb3" exitCode=0 Jul 27 16:56:38.207973 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.207782 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" event={"ID":"2ddbfdf1620eb095b72601b0c1530585","Type":"ContainerDied","Data":"ca299be2b7fc5627c40bfd4724a1b5ba96271089d110392c90c25b2d900ecdb3"} Jul 27 16:56:38.209960 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.209930 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" event={"ID":"1e5ee8caab48bd3dff1dc3efc23e7297","Type":"ContainerStarted","Data":"f7921443b78c26c245bab84e58696b2106fa2a4836a55556142e77b6cba37e07"} Jul 27 16:56:38.212819 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.212761 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" event={"ID":"a3c2d2cc-2246-4e1d-b800-2eea9f7be449","Type":"ContainerStarted","Data":"038ad826f37a008f7efba48d1780a077e9823a96fc07cac1b5fa4fcc30930d10"} Jul 27 16:56:38.214304 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.214239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhnf5" event={"ID":"d9b633ee-da5c-4a20-98d1-ad5688caf314","Type":"ContainerStarted","Data":"1179d06d533d13765b1d27a725419a08088777518270ae57fd3a08bcb81cd54e"} Jul 27 16:56:38.216608 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.216580 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k46vl" event={"ID":"81f91d24-0107-4993-bddc-4d8a906f11ff","Type":"ContainerStarted","Data":"dfead470de8881cb7353d8aed0bd189542b60cb524ee129593a4094e67c04864"} Jul 27 16:56:38.218711 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.218662 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qrq8b" event={"ID":"76fc5ab0-111f-4f67-96d6-a030f5c61865","Type":"ContainerStarted","Data":"99a61c06be911652af2116acfecdf98e530ec7ba9a58193fca02c4d66a76a207"} Jul 27 16:56:38.222382 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.222324 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5gbj" event={"ID":"cd1e1b48-2cc3-4e96-9bec-a23842e66afb","Type":"ContainerStarted","Data":"019c300274a42d353a6c802289f1a363a7cf0d58af3fa4fae641ed04b983ad8e"} Jul 27 16:56:38.226413 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.226341 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerStarted","Data":"0d59145de4a27ee666312306a8202bf8ad037a49f7aaaf96ea84a366f8eedc40"} Jul 27 16:56:38.233322 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.233197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nk6nm" event={"ID":"5bd31b56-43d8-4f43-9a3e-28ffbab38410","Type":"ContainerStarted","Data":"55cb830ed987f0bce8ddce27a2e4e1a43f3ff25d0ba044e563cc48decf65985c"} Jul 27 16:56:38.235713 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.235684 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"1f112c74441d00d64a62062959c76c85078f4d72b1eb43a1b828465c91683574"} Jul 27 16:56:38.248620 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:38.248088 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-54.ec2.internal" podStartSLOduration=2.248057009 podStartE2EDuration="2.248057009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:38.247593974 +0000 UTC m=+4.531753261" watchObservedRunningTime="2026-07-27 16:56:38.248057009 +0000 UTC m=+4.532216297" Jul 27 16:56:39.192890 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:39.192852 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:39.193580 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.192995 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:39.243628 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:39.243591 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" event={"ID":"2ddbfdf1620eb095b72601b0c1530585","Type":"ContainerStarted","Data":"ab5b73ae2f316335c845933fc944ccff3c315bfe456abd67f024e6fc72aba597"} Jul 27 16:56:39.672789 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:39.672746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:39.673113 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.672988 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:56:39.673113 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.673013 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:56:39.673113 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.673027 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:39.673113 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.673090 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:43.673070581 +0000 UTC m=+9.957229849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:39.774049 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:39.773788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:39.774049 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:39.773911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:39.774049 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.773964 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:39.774049 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.774002 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:39.774049 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.774041 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:43.774019082 +0000 UTC m=+10.058178366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:39.774049 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:39.774059 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:43.774051356 +0000 UTC m=+10.058210620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:40.192380 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:40.192302 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:40.192380 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:40.192338 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:40.192636 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:40.192478 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:40.192703 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:40.192648 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:41.193052 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:41.193007 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:41.193589 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:41.193159 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:42.193555 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:42.193525 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:42.194047 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:42.193650 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:42.194047 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:42.193744 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:42.194047 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:42.193893 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:43.193623 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:43.193068 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:43.193623 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.193220 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:43.710882 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:43.710787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:43.711085 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.710948 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:56:43.711085 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.710971 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:56:43.711085 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.710983 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:43.711085 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.711048 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:51.711028924 +0000 UTC m=+17.995188201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:43.811932 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:43.811892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:43.812119 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:43.811959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:43.812119 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.812081 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:43.812119 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.812099 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:43.812298 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.812150 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:51.812129372 +0000 UTC m=+18.096288637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:43.812298 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:43.812171 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:56:51.812160406 +0000 UTC m=+18.096319674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:44.193190 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:44.193153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:44.193397 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:44.193263 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:44.193397 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:44.193358 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:44.193514 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:44.193465 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:45.192883 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:45.192828 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:45.193287 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:45.192965 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:46.194082 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:46.193560 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:46.194082 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:46.193691 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:46.194082 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:46.193917 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:46.194082 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:46.194017 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:47.192019 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:47.191987 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:47.192185 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:47.192108 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:47.272991 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:47.272943 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" event={"ID":"a3c2d2cc-2246-4e1d-b800-2eea9f7be449","Type":"ContainerStarted","Data":"5af01daefee8e34024563bb6ef93cff1c74e6eb4b50b8fc647a55464665e6eb9"} Jul 27 16:56:48.193069 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:48.193032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:48.193069 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:48.193071 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:48.193297 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:48.193174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:48.193352 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:48.193300 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:49.192868 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:49.192826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:49.193305 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:49.192959 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:50.192978 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:50.192942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:50.193475 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:50.192942 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:50.193475 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:50.193083 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:50.193475 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:50.193169 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:51.192899 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:51.192867 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:51.193100 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.192967 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:51.775689 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:51.775655 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:51.775901 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.775796 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:56:51.775901 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.775821 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:56:51.775901 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.775833 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:51.775901 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.775886 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:07.775868958 +0000 UTC m=+34.060028235 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:56:51.876441 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:51.876356 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:51.876618 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:51.876455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:51.876618 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.876526 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:51.876618 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.876567 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:51.876618 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.876602 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:07.876580192 +0000 UTC m=+34.160739464 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:56:51.876836 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:51.876625 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:07.876614151 +0000 UTC m=+34.160773420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:56:52.192403 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:52.192313 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:52.192561 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:52.192461 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:52.192561 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:52.192510 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:52.192660 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:52.192638 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:53.192150 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:53.192119 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:53.192596 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:53.192241 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:54.193145 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:54.193106 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:54.193592 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:54.193210 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:54.193592 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:54.193281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:54.193592 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:54.193395 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:55.192362 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:55.192327 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:55.192560 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:55.192465 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:56.069974 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.069790 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 27 16:56:56.094467 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.094319 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T16:56:56.069971557Z","UUID":"b3faed70-1c96-4c2f-bc00-6816ea2c4172","Handler":null,"Name":"","Endpoint":""} Jul 27 16:56:56.095780 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.095764 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 27 16:56:56.095869 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.095796 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 27 16:56:56.192617 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.192439 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:56.192731 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.192447 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:56.192783 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:56.192719 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:56.192838 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:56.192806 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:56.293972 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.293940 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" event={"ID":"a3c2d2cc-2246-4e1d-b800-2eea9f7be449","Type":"ContainerStarted","Data":"c3fba9f8676972b30210a18f137c8625560fee797d08c928a805d248ae4a5e8a"} Jul 27 16:56:56.295219 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.295195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhnf5" event={"ID":"d9b633ee-da5c-4a20-98d1-ad5688caf314","Type":"ContainerStarted","Data":"c0f8f8e88aaa5164b05271a52e3dfe70be29a1fcda96a9ac5dce72973827e304"} Jul 27 16:56:56.296542 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.296522 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k46vl" event={"ID":"81f91d24-0107-4993-bddc-4d8a906f11ff","Type":"ContainerStarted","Data":"c5cf71101984a138e34545232a367ebb6e565be6516ce2548ec3fec032937e68"} Jul 27 16:56:56.297866 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.297847 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5gbj" event={"ID":"cd1e1b48-2cc3-4e96-9bec-a23842e66afb","Type":"ContainerStarted","Data":"e23234355704209cded927b97f9ca625774a3d2a5a2b629984c0e4da22321e83"} Jul 27 16:56:56.299225 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.299205 2582 generic.go:356] "Generic (PLEG): container finished" podID="e0d78532-02c9-4292-aa11-7f235a427af1" containerID="b66c29b839c0b659c0540365691b66444a65cdc1a76dac2e0a3ef8a7507612d5" exitCode=0 Jul 27 16:56:56.299309 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.299269 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerDied","Data":"b66c29b839c0b659c0540365691b66444a65cdc1a76dac2e0a3ef8a7507612d5"} Jul 27 16:56:56.300759 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.300724 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nk6nm" event={"ID":"5bd31b56-43d8-4f43-9a3e-28ffbab38410","Type":"ContainerStarted","Data":"da99295ed662332a14a894904ae5e3ed341216e7e464257d4d54ea9f168df268"} Jul 27 16:56:56.303314 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.303222 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"6fec7c0a49bd7545ccc7040c4a41bd590a5971a74c1aacab09aa577425951b5c"} Jul 27 16:56:56.303314 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.303282 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"b584f3924debfcd194cbf2ea3b89a207158030f48bca96b3fcc934fe8b18817e"} Jul 27 16:56:56.303458 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.303334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"a95791f2632f9d876fe92a3bf55a7741bcc98ae5274d841bdf0cd946eeab2d05"} Jul 27 16:56:56.303458 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.303411 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"25ef09ea294993fc184eb09cbb1872826dbc78671ac19166cc40c983111d7f66"} Jul 27 16:56:56.304645 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.304623 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-22rdd" event={"ID":"6a607406-2852-40c5-9633-b4c07b3205f0","Type":"ContainerStarted","Data":"06a5137052749154383d82d3f6c3c121abddbd7a92af3dbb1fcee91e019dd3ce"} Jul 27 16:56:56.315371 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.315335 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vhnf5" podStartSLOduration=2.715918367 podStartE2EDuration="21.315308s" totalImagesPullingTime="18.599389633s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:56.315244822 +0000 UTC m=+22.599404106" watchObservedRunningTime="2026-07-27 16:56:56.315308 +0000 UTC m=+22.599467274" Jul 27 16:56:56.315862 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.315837 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-54.ec2.internal" podStartSLOduration=20.315827345 podStartE2EDuration="20.315827345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:39.263351596 +0000 UTC m=+5.547510884" watchObservedRunningTime="2026-07-27 16:56:56.315827345 +0000 UTC m=+22.599986628" Jul 27 16:56:56.389880 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.389780 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nk6nm" podStartSLOduration=2.996940575 podStartE2EDuration="21.389763578s" totalImagesPullingTime="18.392823003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:56.34660148 +0000 UTC m=+22.630760759" watchObservedRunningTime="2026-07-27 16:56:56.389763578 +0000 UTC m=+22.673922912" Jul 27 16:56:56.426859 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.426813 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-22rdd" podStartSLOduration=8.101280184 podStartE2EDuration="22.426796917s" totalImagesPullingTime="14.325516733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:56.426769609 +0000 UTC m=+22.710928892" watchObservedRunningTime="2026-07-27 16:56:56.426796917 +0000 UTC m=+22.710956199" Jul 27 16:56:56.461598 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:56.461550 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v5gbj" podStartSLOduration=12.922258412 podStartE2EDuration="21.461538746s" totalImagesPullingTime="8.539280334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:56.461448354 +0000 UTC m=+22.745607637" watchObservedRunningTime="2026-07-27 16:56:56.461538746 +0000 UTC m=+22.745698028" Jul 27 16:56:57.192745 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.192671 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:57.193404 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:57.192789 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:57.310048 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.310017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"01fd898f85562409eb154926567f1b35bcb252760a7a7e178c3904f41e1effa0"} Jul 27 16:56:57.310167 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.310086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"0ff3ac3ce8faf2d058343474ba19fc3ed3dc4a138845171c9b6b697989cb6b5f"} Jul 27 16:56:57.315078 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.315039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qrq8b" event={"ID":"76fc5ab0-111f-4f67-96d6-a030f5c61865","Type":"ContainerStarted","Data":"01eeacc7cb2dc29965a6290cb55af959ca2873a1328b9f2734a7dab6460f040d"} Jul 27 16:56:57.335707 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.335666 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" podStartSLOduration=3.319124508 podStartE2EDuration="22.335653963s" totalImagesPullingTime="19.016529455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:57.335524602 +0000 UTC m=+23.619683884" watchObservedRunningTime="2026-07-27 16:56:57.335653963 +0000 UTC m=+23.619813246" Jul 27 16:56:57.335938 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.335920 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k46vl" podStartSLOduration=4.73637851 podStartE2EDuration="23.335913627s" totalImagesPullingTime="18.599535117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:56.499644094 +0000 UTC m=+22.783803377" watchObservedRunningTime="2026-07-27 16:56:57.335913627 +0000 UTC m=+23.620072917" Jul 27 16:56:57.351678 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:57.351645 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qrq8b" podStartSLOduration=8.022567504 podStartE2EDuration="22.351632535s" totalImagesPullingTime="14.329065031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:56:57.351436861 +0000 UTC m=+23.635596143" watchObservedRunningTime="2026-07-27 16:56:57.351632535 +0000 UTC m=+23.635791850" Jul 27 16:56:58.192210 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:58.192175 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:56:58.192434 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:58.192261 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:56:58.192434 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:58.192277 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:56:58.192434 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:58.192396 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:56:58.318842 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:58.318806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vzs2h" event={"ID":"a3c2d2cc-2246-4e1d-b800-2eea9f7be449","Type":"ContainerStarted","Data":"9ee61d79662650aa3edbb3539996cebd4a5b2f08222e14f533072b0a3a757166"} Jul 27 16:56:59.192975 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:59.192946 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:56:59.193154 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:56:59.193059 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:56:59.325315 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:56:59.325272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"a0d666f6c9e7f633d68027e4b5f73fe6534e2e2dfa20823fc346fbeba3db4b50"} Jul 27 16:57:00.192446 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:00.192414 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:00.192446 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:00.192439 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:00.192712 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:00.192545 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:57:00.192774 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:00.192704 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:57:00.783727 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:00.783635 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:57:00.784395 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:00.784304 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:57:01.192598 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.192563 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:01.192761 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:01.192667 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:57:01.334572 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.334521 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" event={"ID":"0df410a3-75d4-4715-a942-be270f0d0117","Type":"ContainerStarted","Data":"0c62e1e446042ad6bb86ff8dca24b602c17f5555039f7a3bc1ba817b7deb71e1"} Jul 27 16:57:01.334863 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.334839 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:57:01.335291 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.335080 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:57:01.335446 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.335419 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:57:01.335552 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.335509 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:57:01.335831 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.335809 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nk6nm" Jul 27 16:57:01.351822 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.351795 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:57:01.353369 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.353348 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:57:01.366339 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:01.366296 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" podStartSLOduration=7.689048004 podStartE2EDuration="26.366282362s" totalImagesPullingTime="18.677234358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:01.365922372 +0000 UTC m=+27.650081656" watchObservedRunningTime="2026-07-27 16:57:01.366282362 +0000 UTC m=+27.650441643" Jul 27 16:57:02.193029 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.193005 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:02.193697 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.193006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:02.193697 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:02.193119 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:57:02.193697 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:02.193228 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:57:02.810016 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.809831 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rhgrq"] Jul 27 16:57:02.810157 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.810099 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:02.810223 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:02.810191 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:57:02.813954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.812881 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kb9bl"] Jul 27 16:57:02.813954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.812969 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:02.813954 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:02.813072 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:57:02.816110 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.816025 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bx5ct"] Jul 27 16:57:02.816211 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:02.816114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:02.816265 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:02.816213 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:57:03.341429 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:03.341374 2582 generic.go:356] "Generic (PLEG): container finished" podID="e0d78532-02c9-4292-aa11-7f235a427af1" containerID="75df1be4851e3e33300c24f0d4954f43cf8dced746daea6d8c8d492c813891f7" exitCode=0 Jul 27 16:57:03.341868 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:03.341479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerDied","Data":"75df1be4851e3e33300c24f0d4954f43cf8dced746daea6d8c8d492c813891f7"} Jul 27 16:57:04.192756 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:04.192723 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:04.192945 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:04.192814 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:57:04.192945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:04.192841 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:04.192945 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:04.192932 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:57:04.193057 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:04.192957 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:04.193057 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:04.193036 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:57:05.347459 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:05.347424 2582 generic.go:356] "Generic (PLEG): container finished" podID="e0d78532-02c9-4292-aa11-7f235a427af1" containerID="384393dcf3b6906cc675d2efa0fd75c530244b12e86407bb4988931145b4b81b" exitCode=0 Jul 27 16:57:05.347814 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:05.347496 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerDied","Data":"384393dcf3b6906cc675d2efa0fd75c530244b12e86407bb4988931145b4b81b"} Jul 27 16:57:06.192712 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:06.192677 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:06.192896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:06.192685 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:06.192896 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:06.192790 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:57:06.192896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:06.192823 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:06.193047 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:06.192896 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:57:06.193047 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:06.192951 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:57:07.354661 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:07.354461 2582 generic.go:356] "Generic (PLEG): container finished" podID="e0d78532-02c9-4292-aa11-7f235a427af1" containerID="5b92ccbb300662cc665d47bf9fc05cf5af6c449644704f834fa0fe40f5d49ec3" exitCode=0 Jul 27 16:57:07.355041 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:07.354547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerDied","Data":"5b92ccbb300662cc665d47bf9fc05cf5af6c449644704f834fa0fe40f5d49ec3"} Jul 27 16:57:07.790709 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:07.790609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:07.790855 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.790785 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 16:57:07.790855 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.790807 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 16:57:07.790855 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.790818 2582 projected.go:196] Error preparing data for projected volume kube-api-access-bhhjb for pod openshift-network-diagnostics/network-check-target-rhgrq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:57:07.790953 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.790870 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb podName:b7a1239e-6e14-4054-8b90-63d4d089ee08 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:39.790855105 +0000 UTC m=+66.075014369 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhhjb" (UniqueName: "kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb") pod "network-check-target-rhgrq" (UID: "b7a1239e-6e14-4054-8b90-63d4d089ee08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 16:57:07.891341 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:07.891308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:07.891490 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:07.891351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:07.891490 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.891478 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:57:07.891577 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.891497 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 16:57:07.891577 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.891547 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs podName:0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:39.891528249 +0000 UTC m=+66.175687512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs") pod "network-metrics-daemon-bx5ct" (UID: "0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 16:57:07.891577 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:07.891560 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret podName:ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:39.891554349 +0000 UTC m=+66.175713610 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret") pod "global-pull-secret-syncer-kb9bl" (UID: "ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9") : object "kube-system"/"original-pull-secret" not registered Jul 27 16:57:08.192432 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.192392 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:08.192432 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.192413 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:08.192641 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.192450 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:08.192641 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:08.192584 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kb9bl" podUID="ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9" Jul 27 16:57:08.192720 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:08.192677 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rhgrq" podUID="b7a1239e-6e14-4054-8b90-63d4d089ee08" Jul 27 16:57:08.192759 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:08.192743 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bx5ct" podUID="0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7" Jul 27 16:57:08.523617 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.523539 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-54.ec2.internal" event="NodeReady" Jul 27 16:57:08.523971 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.523669 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 27 16:57:08.572218 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.572182 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-567dl"] Jul 27 16:57:08.590775 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.590749 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-29696"] Jul 27 16:57:08.590935 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.590901 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.595108 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.595082 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 27 16:57:08.596043 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.596021 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rkv78\"" Jul 27 16:57:08.597014 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.596895 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 27 16:57:08.613301 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.613267 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-567dl"] Jul 27 16:57:08.613443 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.613310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-29696"] Jul 27 16:57:08.613443 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.613413 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.617000 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.616787 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 27 16:57:08.617000 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.616796 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 27 16:57:08.617000 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.616903 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-84blb\"" Jul 27 16:57:08.618071 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.618051 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 27 16:57:08.697699 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.697662 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjnc\" (UniqueName: \"kubernetes.io/projected/3df2fcdf-403d-4c07-9337-29ec40b859bd-kube-api-access-fkjnc\") pod \"ingress-canary-29696\" (UID: \"3df2fcdf-403d-4c07-9337-29ec40b859bd\") " pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.697882 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.697734 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3df2fcdf-403d-4c07-9337-29ec40b859bd-cert\") pod \"ingress-canary-29696\" (UID: \"3df2fcdf-403d-4c07-9337-29ec40b859bd\") " pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.697882 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.697818 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-tmp-dir\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.697882 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.697854 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnpwz\" (UniqueName: \"kubernetes.io/projected/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-kube-api-access-mnpwz\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.698017 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.697903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.698017 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.697941 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-config-volume\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.798656 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.798579 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjnc\" (UniqueName: \"kubernetes.io/projected/3df2fcdf-403d-4c07-9337-29ec40b859bd-kube-api-access-fkjnc\") pod \"ingress-canary-29696\" (UID: \"3df2fcdf-403d-4c07-9337-29ec40b859bd\") " pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.798656 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.798638 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3df2fcdf-403d-4c07-9337-29ec40b859bd-cert\") pod \"ingress-canary-29696\" (UID: \"3df2fcdf-403d-4c07-9337-29ec40b859bd\") " pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.798881 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.798688 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-tmp-dir\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.798881 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.798710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnpwz\" (UniqueName: \"kubernetes.io/projected/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-kube-api-access-mnpwz\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.798881 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.798745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.798881 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.798770 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-config-volume\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.799122 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:08.798993 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 16:57:08.799122 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:08.799050 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls podName:a906bcd2-a953-4dc1-802b-ea4f840b3bcb nodeName:}" failed. No retries permitted until 2026-07-27 16:57:09.299031997 +0000 UTC m=+35.583191270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls") pod "dns-default-567dl" (UID: "a906bcd2-a953-4dc1-802b-ea4f840b3bcb") : secret "dns-default-metrics-tls" not found Jul 27 16:57:08.804208 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.804169 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3df2fcdf-403d-4c07-9337-29ec40b859bd-cert\") pod \"ingress-canary-29696\" (UID: \"3df2fcdf-403d-4c07-9337-29ec40b859bd\") " pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.808582 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.808559 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjnc\" (UniqueName: \"kubernetes.io/projected/3df2fcdf-403d-4c07-9337-29ec40b859bd-kube-api-access-fkjnc\") pod \"ingress-canary-29696\" (UID: \"3df2fcdf-403d-4c07-9337-29ec40b859bd\") " pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:08.810444 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.810396 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-tmp-dir\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.810746 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.810721 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-config-volume\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.812634 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.812587 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnpwz\" (UniqueName: \"kubernetes.io/projected/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-kube-api-access-mnpwz\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:08.922962 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:08.922924 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-29696" Jul 27 16:57:09.120058 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:09.120026 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-29696"] Jul 27 16:57:09.124342 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:09.124314 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df2fcdf_403d_4c07_9337_29ec40b859bd.slice/crio-3b27b668e26b34943dd8bbdf4a62961cced55e596e9e0fa9aa0fb2e23fff49a1 WatchSource:0}: Error finding container 3b27b668e26b34943dd8bbdf4a62961cced55e596e9e0fa9aa0fb2e23fff49a1: Status 404 returned error can't find the container with id 3b27b668e26b34943dd8bbdf4a62961cced55e596e9e0fa9aa0fb2e23fff49a1 Jul 27 16:57:09.303053 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:09.303023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:09.303302 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:09.303200 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 16:57:09.303302 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:09.303267 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls podName:a906bcd2-a953-4dc1-802b-ea4f840b3bcb nodeName:}" failed. No retries permitted until 2026-07-27 16:57:10.303249728 +0000 UTC m=+36.587408989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls") pod "dns-default-567dl" (UID: "a906bcd2-a953-4dc1-802b-ea4f840b3bcb") : secret "dns-default-metrics-tls" not found Jul 27 16:57:09.360857 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:09.360816 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-29696" event={"ID":"3df2fcdf-403d-4c07-9337-29ec40b859bd","Type":"ContainerStarted","Data":"3b27b668e26b34943dd8bbdf4a62961cced55e596e9e0fa9aa0fb2e23fff49a1"} Jul 27 16:57:10.197871 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.197832 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:10.197871 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.197864 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:10.198925 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.197869 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:10.203274 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.203251 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 16:57:10.203427 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.203256 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 16:57:10.203655 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.203638 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 16:57:10.203787 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.203770 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g7rsp\"" Jul 27 16:57:10.203871 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.203858 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 16:57:10.203931 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.203903 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5cwf9\"" Jul 27 16:57:10.308823 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:10.308784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:10.308989 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:10.308951 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 16:57:10.309042 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:10.309012 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls podName:a906bcd2-a953-4dc1-802b-ea4f840b3bcb nodeName:}" failed. No retries permitted until 2026-07-27 16:57:12.308997684 +0000 UTC m=+38.593156950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls") pod "dns-default-567dl" (UID: "a906bcd2-a953-4dc1-802b-ea4f840b3bcb") : secret "dns-default-metrics-tls" not found Jul 27 16:57:12.325398 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:12.325347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:12.325846 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:12.325500 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 16:57:12.325846 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:12.325584 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls podName:a906bcd2-a953-4dc1-802b-ea4f840b3bcb nodeName:}" failed. No retries permitted until 2026-07-27 16:57:16.325562202 +0000 UTC m=+42.609721482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls") pod "dns-default-567dl" (UID: "a906bcd2-a953-4dc1-802b-ea4f840b3bcb") : secret "dns-default-metrics-tls" not found Jul 27 16:57:13.374873 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:13.374783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerStarted","Data":"c8aea7a1b36902f859ef5ae4dc4f87577dcab2d8bb36fab44bacb4d6f4ee9c04"} Jul 27 16:57:13.376576 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:13.376548 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-29696" event={"ID":"3df2fcdf-403d-4c07-9337-29ec40b859bd","Type":"ContainerStarted","Data":"f44e04077dfadc695f5d289d51464d17caec6b8a0a8101e6291a05ea633b9bfa"} Jul 27 16:57:13.443267 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:13.443206 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-29696" podStartSLOduration=1.433279634 podStartE2EDuration="5.44318795s" totalImagesPullingTime="4.009908316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:13.442628703 +0000 UTC m=+39.726787989" watchObservedRunningTime="2026-07-27 16:57:13.44318795 +0000 UTC m=+39.727347212" Jul 27 16:57:14.381220 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:14.381187 2582 generic.go:356] "Generic (PLEG): container finished" podID="e0d78532-02c9-4292-aa11-7f235a427af1" containerID="c8aea7a1b36902f859ef5ae4dc4f87577dcab2d8bb36fab44bacb4d6f4ee9c04" exitCode=0 Jul 27 16:57:14.381611 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:14.381272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerDied","Data":"c8aea7a1b36902f859ef5ae4dc4f87577dcab2d8bb36fab44bacb4d6f4ee9c04"} Jul 27 16:57:15.310560 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.310519 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft"] Jul 27 16:57:15.330444 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.330410 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4"] Jul 27 16:57:15.330617 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.330533 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" Jul 27 16:57:15.333703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.333542 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 27 16:57:15.333703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.333588 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 27 16:57:15.333703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.333629 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w8twd\"" Jul 27 16:57:15.344332 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.344309 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft"] Jul 27 16:57:15.344445 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.344342 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4"] Jul 27 16:57:15.344493 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.344445 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" Jul 27 16:57:15.347201 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.347176 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-24446\"" Jul 27 16:57:15.386503 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.386476 2582 generic.go:356] "Generic (PLEG): container finished" podID="e0d78532-02c9-4292-aa11-7f235a427af1" containerID="e290ba5b9155f15c52c07d4adf92ea259bc9840b0146f1cc36d7d17777a75305" exitCode=0 Jul 27 16:57:15.386852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.386525 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerDied","Data":"e290ba5b9155f15c52c07d4adf92ea259bc9840b0146f1cc36d7d17777a75305"} Jul 27 16:57:15.408279 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.408253 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bxswn"] Jul 27 16:57:15.420752 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.420727 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79d8f4d6d7-cbq28"] Jul 27 16:57:15.420860 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.420844 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.424275 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.424255 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rshbc\"" Jul 27 16:57:15.424556 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.424540 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 27 16:57:15.424779 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.424765 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 27 16:57:15.424896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.424878 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 27 16:57:15.425219 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.425202 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 27 16:57:15.440201 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.440173 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 27 16:57:15.448164 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.448135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkd7n\" (UniqueName: \"kubernetes.io/projected/2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd-kube-api-access-gkd7n\") pod \"network-check-source-7d88cb55d8-rhxq4\" (UID: \"2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" Jul 27 16:57:15.448457 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.448430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9sdp\" (UniqueName: \"kubernetes.io/projected/2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85-kube-api-access-h9sdp\") pod \"volume-data-source-validator-549c6d54cf-2lbft\" (UID: \"2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" Jul 27 16:57:15.448886 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.448858 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bxswn"] Jul 27 16:57:15.448989 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.448896 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79d8f4d6d7-cbq28"] Jul 27 16:57:15.449050 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.448986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.451997 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.451974 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 27 16:57:15.452431 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.452410 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9fhsl\"" Jul 27 16:57:15.452491 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.452456 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 27 16:57:15.452805 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.452785 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 27 16:57:15.452881 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.452808 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 27 16:57:15.453032 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.453015 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 27 16:57:15.453108 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.453023 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 27 16:57:15.513017 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.512991 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l"] Jul 27 16:57:15.523061 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.523037 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7"] Jul 27 16:57:15.523201 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.523181 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.526594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.526152 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 27 16:57:15.526594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.526239 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 27 16:57:15.526594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.526269 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 27 16:57:15.526594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.526446 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qzphw\"" Jul 27 16:57:15.526594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.526570 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 27 16:57:15.535265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.535232 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l"] Jul 27 16:57:15.535405 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.535272 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7"] Jul 27 16:57:15.535405 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.535353 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.538170 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.538151 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 27 16:57:15.538304 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.538212 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 27 16:57:15.538406 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.538390 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 27 16:57:15.538466 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.538413 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-cfb8x\"" Jul 27 16:57:15.538466 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.538434 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 27 16:57:15.548749 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.548725 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.548876 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.548788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9sdp\" (UniqueName: \"kubernetes.io/projected/2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85-kube-api-access-h9sdp\") pod \"volume-data-source-validator-549c6d54cf-2lbft\" (UID: \"2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" Jul 27 16:57:15.548876 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.548816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.548876 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.548841 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-service-ca-bundle\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.548996 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.548886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5md6z\" (UniqueName: \"kubernetes.io/projected/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-kube-api-access-5md6z\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.549034 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.548998 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkd7n\" (UniqueName: \"kubernetes.io/projected/2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd-kube-api-access-gkd7n\") pod \"network-check-source-7d88cb55d8-rhxq4\" (UID: \"2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" Jul 27 16:57:15.549034 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549023 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-stats-auth\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.549093 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkrrz\" (UniqueName: \"kubernetes.io/projected/66329ae6-c737-4391-a021-514669245bc8-kube-api-access-vkrrz\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.549093 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549073 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-tmp\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.549153 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-serving-cert\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.549153 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549142 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-snapshots\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.549209 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549188 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.549265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.549251 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-default-certificate\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.558861 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.558836 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkd7n\" (UniqueName: \"kubernetes.io/projected/2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd-kube-api-access-gkd7n\") pod \"network-check-source-7d88cb55d8-rhxq4\" (UID: \"2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" Jul 27 16:57:15.558966 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.558911 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9sdp\" (UniqueName: \"kubernetes.io/projected/2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85-kube-api-access-h9sdp\") pod \"volume-data-source-validator-549c6d54cf-2lbft\" (UID: \"2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" Jul 27 16:57:15.639723 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.639686 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" Jul 27 16:57:15.650021 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.649990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.650127 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650034 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcp2n\" (UniqueName: \"kubernetes.io/projected/369747a7-45ae-4fb0-ae48-ca89ea345ec7-kube-api-access-wcp2n\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.650127 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650104 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650249 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650131 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-service-ca-bundle\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650249 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:15.650159 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:16.150142438 +0000 UTC m=+42.434301702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : configmap references non-existent config key: service-ca.crt Jul 27 16:57:15.650360 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650253 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5md6z\" (UniqueName: \"kubernetes.io/projected/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-kube-api-access-5md6z\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650360 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650289 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-stats-auth\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.650360 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkrrz\" (UniqueName: \"kubernetes.io/projected/66329ae6-c737-4391-a021-514669245bc8-kube-api-access-vkrrz\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.650360 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650390 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-tmp\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-serving-cert\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4hs9\" (UniqueName: \"kubernetes.io/projected/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-kube-api-access-f4hs9\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650496 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/369747a7-45ae-4fb0-ae48-ca89ea345ec7-serving-cert\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650626 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-snapshots\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650657 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/369747a7-45ae-4fb0-ae48-ca89ea345ec7-config\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650689 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-default-certificate\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-tmp\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.650782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650771 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.651297 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.650788 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-service-ca-bundle\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.651297 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:15.650885 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 16:57:15.651297 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:15.651085 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:16.151068196 +0000 UTC m=+42.435227464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : secret "router-metrics-certs-default" not found Jul 27 16:57:15.651297 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.651212 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-snapshots\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.651297 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.651219 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.652279 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.652260 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" Jul 27 16:57:15.653128 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.653112 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-default-certificate\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.653287 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.653268 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-serving-cert\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.653337 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.653274 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-stats-auth\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.660050 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.660029 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5md6z\" (UniqueName: \"kubernetes.io/projected/c9d28086-fbfc-4d08-9d3d-1db5193d3c1f-kube-api-access-5md6z\") pod \"insights-operator-5885fdb759-bxswn\" (UID: \"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f\") " pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.660199 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.660179 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkrrz\" (UniqueName: \"kubernetes.io/projected/66329ae6-c737-4391-a021-514669245bc8-kube-api-access-vkrrz\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:15.740666 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.740627 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-bxswn" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.752063 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/369747a7-45ae-4fb0-ae48-ca89ea345ec7-serving-cert\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.752133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/369747a7-45ae-4fb0-ae48-ca89ea345ec7-config\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.752217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.752261 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcp2n\" (UniqueName: \"kubernetes.io/projected/369747a7-45ae-4fb0-ae48-ca89ea345ec7-kube-api-access-wcp2n\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.752344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.752402 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4hs9\" (UniqueName: \"kubernetes.io/projected/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-kube-api-access-f4hs9\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.753676 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.753614 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.754092 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.753718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/369747a7-45ae-4fb0-ae48-ca89ea345ec7-config\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.756005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.755979 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.756122 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.755979 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/369747a7-45ae-4fb0-ae48-ca89ea345ec7-serving-cert\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.762852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.762555 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcp2n\" (UniqueName: \"kubernetes.io/projected/369747a7-45ae-4fb0-ae48-ca89ea345ec7-kube-api-access-wcp2n\") pod \"service-ca-operator-5d6c75f84f-l7hs7\" (UID: \"369747a7-45ae-4fb0-ae48-ca89ea345ec7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.764031 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.764009 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4hs9\" (UniqueName: \"kubernetes.io/projected/6c84d615-a6b0-488c-9dcd-3fcc11bbea80-kube-api-access-f4hs9\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l\" (UID: \"6c84d615-a6b0-488c-9dcd-3fcc11bbea80\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.808749 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.808672 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft"] Jul 27 16:57:15.813308 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:15.813274 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5f0b6d_2ba7_487f_8594_a2a6e9e5aa85.slice/crio-7cc77a6f13ab8dba00f062986b8a71bcaa7ae2949564c7f0f534173a8be80bdd WatchSource:0}: Error finding container 7cc77a6f13ab8dba00f062986b8a71bcaa7ae2949564c7f0f534173a8be80bdd: Status 404 returned error can't find the container with id 7cc77a6f13ab8dba00f062986b8a71bcaa7ae2949564c7f0f534173a8be80bdd Jul 27 16:57:15.834530 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.834020 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" Jul 27 16:57:15.845359 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.844557 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" Jul 27 16:57:15.850308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.850162 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4"] Jul 27 16:57:15.853083 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:15.853047 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7e4a4d_4b0d_410f_9d34_c8cc5d1824fd.slice/crio-cb05561bbe50369508e0c116901645a9d908d96755a5b685eb942a2a60757f2c WatchSource:0}: Error finding container cb05561bbe50369508e0c116901645a9d908d96755a5b685eb942a2a60757f2c: Status 404 returned error can't find the container with id cb05561bbe50369508e0c116901645a9d908d96755a5b685eb942a2a60757f2c Jul 27 16:57:15.908176 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.908148 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-bxswn"] Jul 27 16:57:15.910282 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:15.910246 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d28086_fbfc_4d08_9d3d_1db5193d3c1f.slice/crio-82c6a7bad51a464857114aceac1c5c0fe26b7bbca67be118639850e3c9663d08 WatchSource:0}: Error finding container 82c6a7bad51a464857114aceac1c5c0fe26b7bbca67be118639850e3c9663d08: Status 404 returned error can't find the container with id 82c6a7bad51a464857114aceac1c5c0fe26b7bbca67be118639850e3c9663d08 Jul 27 16:57:15.996821 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:15.996745 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l"] Jul 27 16:57:16.001007 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:16.000979 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c84d615_a6b0_488c_9dcd_3fcc11bbea80.slice/crio-b2f44947be451ef13cf072ca1fd9e3f4f89e598b3d9acf524258995fd605c6cd WatchSource:0}: Error finding container b2f44947be451ef13cf072ca1fd9e3f4f89e598b3d9acf524258995fd605c6cd: Status 404 returned error can't find the container with id b2f44947be451ef13cf072ca1fd9e3f4f89e598b3d9acf524258995fd605c6cd Jul 27 16:57:16.041617 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.041589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7"] Jul 27 16:57:16.044923 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:16.044895 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369747a7_45ae_4fb0_ae48_ca89ea345ec7.slice/crio-dda58684e9b98fe5cbd1c7edadf21f8017fd672330a6ca8d36259d0c7525e5e9 WatchSource:0}: Error finding container dda58684e9b98fe5cbd1c7edadf21f8017fd672330a6ca8d36259d0c7525e5e9: Status 404 returned error can't find the container with id dda58684e9b98fe5cbd1c7edadf21f8017fd672330a6ca8d36259d0c7525e5e9 Jul 27 16:57:16.155437 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.155323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:16.155437 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.155407 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:16.155654 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:16.155486 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 16:57:16.155654 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:16.155546 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:17.15553132 +0000 UTC m=+43.439690584 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : secret "router-metrics-certs-default" not found Jul 27 16:57:16.155654 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:16.155580 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:17.15556103 +0000 UTC m=+43.439720305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : configmap references non-existent config key: service-ca.crt Jul 27 16:57:16.357318 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.357272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:16.357532 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:16.357433 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 16:57:16.357532 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:16.357507 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls podName:a906bcd2-a953-4dc1-802b-ea4f840b3bcb nodeName:}" failed. No retries permitted until 2026-07-27 16:57:24.357486843 +0000 UTC m=+50.641646116 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls") pod "dns-default-567dl" (UID: "a906bcd2-a953-4dc1-802b-ea4f840b3bcb") : secret "dns-default-metrics-tls" not found Jul 27 16:57:16.390558 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.390510 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bxswn" event={"ID":"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f","Type":"ContainerStarted","Data":"82c6a7bad51a464857114aceac1c5c0fe26b7bbca67be118639850e3c9663d08"} Jul 27 16:57:16.391782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.391744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" event={"ID":"2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd","Type":"ContainerStarted","Data":"cb05561bbe50369508e0c116901645a9d908d96755a5b685eb942a2a60757f2c"} Jul 27 16:57:16.395644 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.395620 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" event={"ID":"e0d78532-02c9-4292-aa11-7f235a427af1","Type":"ContainerStarted","Data":"7ba4fb2a2b3dbf030343bb33e0b0ff9120872021e2932a952591d59d6e763d9f"} Jul 27 16:57:16.396909 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.396882 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" event={"ID":"369747a7-45ae-4fb0-ae48-ca89ea345ec7","Type":"ContainerStarted","Data":"dda58684e9b98fe5cbd1c7edadf21f8017fd672330a6ca8d36259d0c7525e5e9"} Jul 27 16:57:16.398049 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.398026 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" event={"ID":"6c84d615-a6b0-488c-9dcd-3fcc11bbea80","Type":"ContainerStarted","Data":"b2f44947be451ef13cf072ca1fd9e3f4f89e598b3d9acf524258995fd605c6cd"} Jul 27 16:57:16.399178 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.399157 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" event={"ID":"2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85","Type":"ContainerStarted","Data":"7cc77a6f13ab8dba00f062986b8a71bcaa7ae2949564c7f0f534173a8be80bdd"} Jul 27 16:57:16.421464 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:16.421361 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s9pk6" podStartSLOduration=19.322249326 podStartE2EDuration="42.42134735s" totalImagesPullingTime="23.099098024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:16.419808922 +0000 UTC m=+42.703968205" watchObservedRunningTime="2026-07-27 16:57:16.42134735 +0000 UTC m=+42.705506634" Jul 27 16:57:17.165791 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:17.165417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:17.165791 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:17.165564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:17.165791 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:17.165670 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:19.165651503 +0000 UTC m=+45.449810765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : configmap references non-existent config key: service-ca.crt Jul 27 16:57:17.165791 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:17.165716 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 16:57:17.165791 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:17.165769 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:19.165754433 +0000 UTC m=+45.449913695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : secret "router-metrics-certs-default" not found Jul 27 16:57:19.183436 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:19.183398 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:19.183888 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:19.183456 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:19.183888 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:19.183570 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 16:57:19.183888 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:19.183574 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:23.183555902 +0000 UTC m=+49.467715163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : configmap references non-existent config key: service-ca.crt Jul 27 16:57:19.183888 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:19.183663 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:23.183644414 +0000 UTC m=+49.467803678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : secret "router-metrics-certs-default" not found Jul 27 16:57:21.415876 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:21.415783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" event={"ID":"2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85","Type":"ContainerStarted","Data":"acbc56092e63609ce85d59078a1e7e906eb56db969e2585ef335c6b9157b7505"} Jul 27 16:57:21.417167 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:21.417143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" event={"ID":"6c84d615-a6b0-488c-9dcd-3fcc11bbea80","Type":"ContainerStarted","Data":"0ff75e3146d4c145afd8fc7320a9d8479d31de0ed0196df87278b0edd9c3f4ff"} Jul 27 16:57:21.435558 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:21.435512 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2lbft" podStartSLOduration=1.100852407 podStartE2EDuration="6.435497628s" totalImagesPullingTime="5.334645221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:21.434822386 +0000 UTC m=+47.718981669" watchObservedRunningTime="2026-07-27 16:57:21.435497628 +0000 UTC m=+47.719656910" Jul 27 16:57:21.457620 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:21.457574 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" podStartSLOduration=1.24975509 podStartE2EDuration="6.457563463s" totalImagesPullingTime="5.207808373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:21.45689679 +0000 UTC m=+47.741056074" watchObservedRunningTime="2026-07-27 16:57:21.457563463 +0000 UTC m=+47.741722745" Jul 27 16:57:22.422122 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.422087 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bxswn" event={"ID":"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f","Type":"ContainerStarted","Data":"351a5520b4144ea77506d9aa6f97315286467301c6ef4329c21249a88bec8929"} Jul 27 16:57:22.423788 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.423747 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" event={"ID":"2d7e4a4d-4b0d-410f-9d34-c8cc5d1824fd","Type":"ContainerStarted","Data":"807a970329dff871f249551e87740caa6375c36f20f34757393e8cfa58e3ad01"} Jul 27 16:57:22.425267 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.425242 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" event={"ID":"369747a7-45ae-4fb0-ae48-ca89ea345ec7","Type":"ContainerStarted","Data":"f096b96bd4e54d9c1c04561aae9007df5c3ce88fe9d9bc1301dc1a13fca779a2"} Jul 27 16:57:22.443459 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.443408 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-bxswn" podStartSLOduration=2.138834178 podStartE2EDuration="7.443392045s" totalImagesPullingTime="5.304557867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:22.442837338 +0000 UTC m=+48.726996624" watchObservedRunningTime="2026-07-27 16:57:22.443392045 +0000 UTC m=+48.727551319" Jul 27 16:57:22.479330 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.479273 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" podStartSLOduration=2.012310988 podStartE2EDuration="7.479257894s" totalImagesPullingTime="5.466946906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:22.478834553 +0000 UTC m=+48.762993835" watchObservedRunningTime="2026-07-27 16:57:22.479257894 +0000 UTC m=+48.763417178" Jul 27 16:57:22.479774 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.479741 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-rhxq4" podStartSLOduration=2.125259344 podStartE2EDuration="7.479732206s" totalImagesPullingTime="5.354472862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:22.461729715 +0000 UTC m=+48.745888999" watchObservedRunningTime="2026-07-27 16:57:22.479732206 +0000 UTC m=+48.763891490" Jul 27 16:57:22.626413 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.626364 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8w66f"] Jul 27 16:57:22.628932 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.628912 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.632700 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.632478 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bzm7g\"" Jul 27 16:57:22.632700 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.632501 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 27 16:57:22.632879 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.632734 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 27 16:57:22.643834 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.643807 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8w66f"] Jul 27 16:57:22.717632 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.717541 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e90fe999-65f1-4595-96fb-baeceab626ba-data-volume\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.717632 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.717586 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e90fe999-65f1-4595-96fb-baeceab626ba-crio-socket\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.717632 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.717612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.717864 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.717669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj67b\" (UniqueName: \"kubernetes.io/projected/e90fe999-65f1-4595-96fb-baeceab626ba-kube-api-access-wj67b\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.717864 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.717755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819009 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.818974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819009 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.819010 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj67b\" (UniqueName: \"kubernetes.io/projected/e90fe999-65f1-4595-96fb-baeceab626ba-kube-api-access-wj67b\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819257 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:22.819113 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 16:57:22.819257 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.819148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819257 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:22.819176 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:23.319159872 +0000 UTC m=+49.603319133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : configmap "kube-rbac-proxy" not found Jul 27 16:57:22.819257 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:22.819246 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 16:57:22.819496 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.819274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e90fe999-65f1-4595-96fb-baeceab626ba-data-volume\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819496 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:22.819313 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:23.31929379 +0000 UTC m=+49.603453062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : secret "insights-runtime-extractor-tls" not found Jul 27 16:57:22.819496 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.819363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e90fe999-65f1-4595-96fb-baeceab626ba-crio-socket\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819626 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.819607 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e90fe999-65f1-4595-96fb-baeceab626ba-data-volume\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.819660 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.819604 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e90fe999-65f1-4595-96fb-baeceab626ba-crio-socket\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:22.833909 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:22.833882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj67b\" (UniqueName: \"kubernetes.io/projected/e90fe999-65f1-4595-96fb-baeceab626ba-kube-api-access-wj67b\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:23.222835 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:23.222796 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:23.223042 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:23.222918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:23.223042 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.222970 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:31.222952325 +0000 UTC m=+57.507111586 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : configmap references non-existent config key: service-ca.crt Jul 27 16:57:23.223042 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.223036 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 16:57:23.223150 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.223101 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:31.223086357 +0000 UTC m=+57.507245632 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : secret "router-metrics-certs-default" not found Jul 27 16:57:23.323485 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:23.323447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:23.323644 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:23.323560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:23.323644 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.323602 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 16:57:23.323644 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.323630 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 16:57:23.323737 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.323677 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:24.32365792 +0000 UTC m=+50.607817182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : secret "insights-runtime-extractor-tls" not found Jul 27 16:57:23.323737 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:23.323693 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:24.323686328 +0000 UTC m=+50.607845588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : configmap "kube-rbac-proxy" not found Jul 27 16:57:24.332723 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:24.332687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:24.333082 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:24.332747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:24.333082 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:24.332816 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 16:57:24.333082 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:24.332863 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 16:57:24.333082 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:24.332888 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:26.33286807 +0000 UTC m=+52.617027333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : configmap "kube-rbac-proxy" not found Jul 27 16:57:24.333082 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:24.332907 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:26.332897354 +0000 UTC m=+52.617056616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : secret "insights-runtime-extractor-tls" not found Jul 27 16:57:24.433836 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:24.433806 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:24.434004 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:24.433957 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 16:57:24.434062 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:24.434023 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls podName:a906bcd2-a953-4dc1-802b-ea4f840b3bcb nodeName:}" failed. No retries permitted until 2026-07-27 16:57:40.434006207 +0000 UTC m=+66.718165468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls") pod "dns-default-567dl" (UID: "a906bcd2-a953-4dc1-802b-ea4f840b3bcb") : secret "dns-default-metrics-tls" not found Jul 27 16:57:24.577060 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:24.577036 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhnf5_d9b633ee-da5c-4a20-98d1-ad5688caf314/dns-node-resolver/0.log" Jul 27 16:57:25.577416 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:25.577370 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5gbj_cd1e1b48-2cc3-4e96-9bec-a23842e66afb/node-ca/0.log" Jul 27 16:57:25.980800 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:25.980771 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-29696_3df2fcdf-403d-4c07-9337-29ec40b859bd/serve-healthcheck-canary/0.log" Jul 27 16:57:26.351102 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:26.351072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:26.351281 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:26.351129 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:26.351281 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:26.351203 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 16:57:26.351281 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:26.351235 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 16:57:26.351281 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:26.351270 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:30.351254377 +0000 UTC m=+56.635413638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : configmap "kube-rbac-proxy" not found Jul 27 16:57:26.351506 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:26.351338 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:30.351332068 +0000 UTC m=+56.635491328 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : secret "insights-runtime-extractor-tls" not found Jul 27 16:57:30.379299 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:30.379259 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:30.379773 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:30.379324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:30.379773 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:30.379436 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 16:57:30.379773 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:30.379510 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:38.379493333 +0000 UTC m=+64.663652594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : configmap "kube-rbac-proxy" not found Jul 27 16:57:30.379773 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:30.379507 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 16:57:30.379773 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:30.379538 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:38.379531752 +0000 UTC m=+64.663691012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : secret "insights-runtime-extractor-tls" not found Jul 27 16:57:31.285677 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:31.285624 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:31.285873 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:31.285698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:31.285873 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:31.285774 2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 16:57:31.285873 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:31.285858 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:47.285837735 +0000 UTC m=+73.569996998 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : configmap references non-existent config key: service-ca.crt Jul 27 16:57:31.286003 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:31.285880 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs podName:66329ae6-c737-4391-a021-514669245bc8 nodeName:}" failed. No retries permitted until 2026-07-27 16:57:47.285870501 +0000 UTC m=+73.570029762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs") pod "router-default-79d8f4d6d7-cbq28" (UID: "66329ae6-c737-4391-a021-514669245bc8") : secret "router-metrics-certs-default" not found Jul 27 16:57:33.352393 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:33.352349 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lzcb8" Jul 27 16:57:38.441103 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:38.441066 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:38.441636 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:38.441174 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:38.441636 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:38.441232 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 16:57:38.441636 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:38.441317 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls podName:e90fe999-65f1-4595-96fb-baeceab626ba nodeName:}" failed. No retries permitted until 2026-07-27 16:57:54.441298816 +0000 UTC m=+80.725458080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8w66f" (UID: "e90fe999-65f1-4595-96fb-baeceab626ba") : secret "insights-runtime-extractor-tls" not found Jul 27 16:57:38.441745 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:38.441671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e90fe999-65f1-4595-96fb-baeceab626ba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:39.853968 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.853933 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:39.856412 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.856393 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhhjb\" (UniqueName: \"kubernetes.io/projected/b7a1239e-6e14-4054-8b90-63d4d089ee08-kube-api-access-bhhjb\") pod \"network-check-target-rhgrq\" (UID: \"b7a1239e-6e14-4054-8b90-63d4d089ee08\") " pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:39.921508 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.921476 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5cwf9\"" Jul 27 16:57:39.929082 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.929039 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:39.954572 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.954544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:39.954707 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.954643 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:39.957469 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.957439 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 16:57:39.957609 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.957592 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 16:57:39.967565 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.967497 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9-original-pull-secret\") pod \"global-pull-secret-syncer-kb9bl\" (UID: \"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9\") " pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:39.967679 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:39.967558 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7-metrics-certs\") pod \"network-metrics-daemon-bx5ct\" (UID: \"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7\") " pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:40.052741 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.052711 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rhgrq"] Jul 27 16:57:40.055610 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:40.055584 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a1239e_6e14_4054_8b90_63d4d089ee08.slice/crio-a82869ce7983f5371ab36b5b0bd54f29e9c2c989c67bc6ec472c92c974bd9c4f WatchSource:0}: Error finding container a82869ce7983f5371ab36b5b0bd54f29e9c2c989c67bc6ec472c92c974bd9c4f: Status 404 returned error can't find the container with id a82869ce7983f5371ab36b5b0bd54f29e9c2c989c67bc6ec472c92c974bd9c4f Jul 27 16:57:40.211219 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.211183 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kb9bl" Jul 27 16:57:40.227082 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.227062 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g7rsp\"" Jul 27 16:57:40.234836 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.234813 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bx5ct" Jul 27 16:57:40.361144 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.361070 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kb9bl"] Jul 27 16:57:40.364056 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:40.364032 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7d45c4_f8fe_4ae6_87fb_05dd5a5196c9.slice/crio-1040562f83b2b1a75ff1ee9b5ff9a46f431f5f84b003d2a1cebf9162013c5079 WatchSource:0}: Error finding container 1040562f83b2b1a75ff1ee9b5ff9a46f431f5f84b003d2a1cebf9162013c5079: Status 404 returned error can't find the container with id 1040562f83b2b1a75ff1ee9b5ff9a46f431f5f84b003d2a1cebf9162013c5079 Jul 27 16:57:40.384341 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.384314 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bx5ct"] Jul 27 16:57:40.387044 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:40.387019 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f9ce373_0cf0_44c3_be0f_d2ad3e2b73b7.slice/crio-daf5958bde340d8e909ac73ac3b003b0cae8a09eab100be764d0b9e306249653 WatchSource:0}: Error finding container daf5958bde340d8e909ac73ac3b003b0cae8a09eab100be764d0b9e306249653: Status 404 returned error can't find the container with id daf5958bde340d8e909ac73ac3b003b0cae8a09eab100be764d0b9e306249653 Jul 27 16:57:40.458171 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.458136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:40.460343 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.460324 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a906bcd2-a953-4dc1-802b-ea4f840b3bcb-metrics-tls\") pod \"dns-default-567dl\" (UID: \"a906bcd2-a953-4dc1-802b-ea4f840b3bcb\") " pod="openshift-dns/dns-default-567dl" Jul 27 16:57:40.479210 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.479170 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kb9bl" event={"ID":"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9","Type":"ContainerStarted","Data":"1040562f83b2b1a75ff1ee9b5ff9a46f431f5f84b003d2a1cebf9162013c5079"} Jul 27 16:57:40.480425 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.480404 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rhgrq" event={"ID":"b7a1239e-6e14-4054-8b90-63d4d089ee08","Type":"ContainerStarted","Data":"2dbfbcb948bb42fdf265d5f4233a13e1d7e2befe6988d9b76c908d2cb76fab29"} Jul 27 16:57:40.480502 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.480443 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rhgrq" event={"ID":"b7a1239e-6e14-4054-8b90-63d4d089ee08","Type":"ContainerStarted","Data":"a82869ce7983f5371ab36b5b0bd54f29e9c2c989c67bc6ec472c92c974bd9c4f"} Jul 27 16:57:40.480502 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.480484 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:57:40.481511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.481493 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx5ct" event={"ID":"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7","Type":"ContainerStarted","Data":"daf5958bde340d8e909ac73ac3b003b0cae8a09eab100be764d0b9e306249653"} Jul 27 16:57:40.499729 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.499688 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rhgrq" podStartSLOduration=66.499676038 podStartE2EDuration="1m6.499676038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:40.498930739 +0000 UTC m=+66.783090024" watchObservedRunningTime="2026-07-27 16:57:40.499676038 +0000 UTC m=+66.783835299" Jul 27 16:57:40.706145 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.706070 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rkv78\"" Jul 27 16:57:40.713631 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.713610 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-567dl" Jul 27 16:57:40.838283 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:40.838237 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-567dl"] Jul 27 16:57:40.841111 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:40.841088 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda906bcd2_a953_4dc1_802b_ea4f840b3bcb.slice/crio-a4ecd7dffb56f2db400ac08bdd9ef1b494be82f7e66a7d313582b7f78882695a WatchSource:0}: Error finding container a4ecd7dffb56f2db400ac08bdd9ef1b494be82f7e66a7d313582b7f78882695a: Status 404 returned error can't find the container with id a4ecd7dffb56f2db400ac08bdd9ef1b494be82f7e66a7d313582b7f78882695a Jul 27 16:57:41.487059 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:41.487020 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-567dl" event={"ID":"a906bcd2-a953-4dc1-802b-ea4f840b3bcb","Type":"ContainerStarted","Data":"a4ecd7dffb56f2db400ac08bdd9ef1b494be82f7e66a7d313582b7f78882695a"} Jul 27 16:57:42.493260 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:42.493220 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx5ct" event={"ID":"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7","Type":"ContainerStarted","Data":"0ac9ab4ee0a903971072ab811209ba1c1746e5c6fd8f92f01fb0b7e068e8cda6"} Jul 27 16:57:42.493714 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:42.493290 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bx5ct" event={"ID":"0f9ce373-0cf0-44c3-be0f-d2ad3e2b73b7","Type":"ContainerStarted","Data":"77ff67c14d3faf39e60b31e3d4159611cb2d51e58b26f44d6bcd8cc779f49a69"} Jul 27 16:57:42.515535 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:42.515303 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bx5ct" podStartSLOduration=66.294890815 podStartE2EDuration="1m7.515280494s" totalImagesPullingTime="1.220389679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:42.512572141 +0000 UTC m=+68.796731425" watchObservedRunningTime="2026-07-27 16:57:42.515280494 +0000 UTC m=+68.799439780" Jul 27 16:57:43.499559 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:43.499521 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-567dl" event={"ID":"a906bcd2-a953-4dc1-802b-ea4f840b3bcb","Type":"ContainerStarted","Data":"97dcbe6ab38e0f7c14d8519410ae158ea89d3ffb217eeba1ef4d25e1770201ac"} Jul 27 16:57:43.500052 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:43.499574 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-567dl" event={"ID":"a906bcd2-a953-4dc1-802b-ea4f840b3bcb","Type":"ContainerStarted","Data":"5bb024d624fea1da0b95f49df5d6e3a77bcf642bab20816df5e2978b88910ac4"} Jul 27 16:57:43.521091 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:43.521030 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-567dl" podStartSLOduration=33.725481463 podStartE2EDuration="35.520998169s" totalImagesPullingTime="1.795516706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:43.520932532 +0000 UTC m=+69.805091818" watchObservedRunningTime="2026-07-27 16:57:43.520998169 +0000 UTC m=+69.805157453" Jul 27 16:57:44.503451 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:44.503405 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-567dl" Jul 27 16:57:46.510099 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:46.510064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kb9bl" event={"ID":"ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9","Type":"ContainerStarted","Data":"831f1b3823d65559f2b19a2692d9e9974fc02a67b567d0d28aaa8b62c411b87e"} Jul 27 16:57:47.316679 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.316645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:47.316880 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.316687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:47.317295 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.317270 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66329ae6-c737-4391-a021-514669245bc8-service-ca-bundle\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:47.319163 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.319131 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66329ae6-c737-4391-a021-514669245bc8-metrics-certs\") pod \"router-default-79d8f4d6d7-cbq28\" (UID: \"66329ae6-c737-4391-a021-514669245bc8\") " pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:47.563176 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.563140 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9fhsl\"" Jul 27 16:57:47.570805 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.570742 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:47.722244 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.722188 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kb9bl" podStartSLOduration=67.589451702 podStartE2EDuration="1m12.722171101s" totalImagesPullingTime="5.132719399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:46.533125584 +0000 UTC m=+72.817284867" watchObservedRunningTime="2026-07-27 16:57:47.722171101 +0000 UTC m=+74.006330386" Jul 27 16:57:47.723205 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:47.723175 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79d8f4d6d7-cbq28"] Jul 27 16:57:47.726726 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:47.726703 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66329ae6_c737_4391_a021_514669245bc8.slice/crio-bbb9f40dcf115810e8731680952beb8aff7b165126c91c338b4399847910861f WatchSource:0}: Error finding container bbb9f40dcf115810e8731680952beb8aff7b165126c91c338b4399847910861f: Status 404 returned error can't find the container with id bbb9f40dcf115810e8731680952beb8aff7b165126c91c338b4399847910861f Jul 27 16:57:48.518054 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:48.518022 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" event={"ID":"66329ae6-c737-4391-a021-514669245bc8","Type":"ContainerStarted","Data":"0fe9f0f1216999e7ff6fef17bebd8f79201a0ad551f0a99dc2e508ba3ff0da1d"} Jul 27 16:57:48.518223 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:48.518069 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" event={"ID":"66329ae6-c737-4391-a021-514669245bc8","Type":"ContainerStarted","Data":"bbb9f40dcf115810e8731680952beb8aff7b165126c91c338b4399847910861f"} Jul 27 16:57:48.547575 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:48.547520 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" podStartSLOduration=33.547505055 podStartE2EDuration="33.547505055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:48.547482967 +0000 UTC m=+74.831642249" watchObservedRunningTime="2026-07-27 16:57:48.547505055 +0000 UTC m=+74.831664338" Jul 27 16:57:48.571453 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:48.571417 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:48.573929 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:48.573907 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:49.520908 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.520879 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:49.522143 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.522123 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79d8f4d6d7-cbq28" Jul 27 16:57:49.900572 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.900541 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dnr46"] Jul 27 16:57:49.927721 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.927683 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dnr46"] Jul 27 16:57:49.927864 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.927827 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:49.930797 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.930777 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 27 16:57:49.931036 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.931020 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 27 16:57:49.931118 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.931029 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-m527b\"" Jul 27 16:57:49.994314 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:49.994285 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b8c8d4fdc-xhswd"] Jul 27 16:57:50.020721 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.020694 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8c8d4fdc-xhswd"] Jul 27 16:57:50.020866 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.020806 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.023727 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.023697 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lh2hl\"" Jul 27 16:57:50.024038 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.024020 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 27 16:57:50.024098 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.024024 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 27 16:57:50.024598 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.024582 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 27 16:57:50.036141 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.036119 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3cb439a8-fda2-463a-8abb-75b3bb645314-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dnr46\" (UID: \"3cb439a8-fda2-463a-8abb-75b3bb645314\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.036246 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.036182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3cb439a8-fda2-463a-8abb-75b3bb645314-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dnr46\" (UID: \"3cb439a8-fda2-463a-8abb-75b3bb645314\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.038265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.038245 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 27 16:57:50.136943 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.136848 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3cb439a8-fda2-463a-8abb-75b3bb645314-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dnr46\" (UID: \"3cb439a8-fda2-463a-8abb-75b3bb645314\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.136943 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.136900 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eff295e-1fad-4966-9d70-769c896d0561-image-registry-private-configuration\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.136943 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.136928 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5cqh\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-kube-api-access-v5cqh\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137182 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.136970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-registry-tls\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137182 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137031 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eff295e-1fad-4966-9d70-769c896d0561-ca-trust-extracted\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137182 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137061 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-bound-sa-token\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137182 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137080 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eff295e-1fad-4966-9d70-769c896d0561-installation-pull-secrets\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137182 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eff295e-1fad-4966-9d70-769c896d0561-trusted-ca\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137182 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3cb439a8-fda2-463a-8abb-75b3bb645314-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dnr46\" (UID: \"3cb439a8-fda2-463a-8abb-75b3bb645314\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.137498 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137244 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eff295e-1fad-4966-9d70-769c896d0561-registry-certificates\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.137852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.137830 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3cb439a8-fda2-463a-8abb-75b3bb645314-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dnr46\" (UID: \"3cb439a8-fda2-463a-8abb-75b3bb645314\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.139272 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.139256 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3cb439a8-fda2-463a-8abb-75b3bb645314-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dnr46\" (UID: \"3cb439a8-fda2-463a-8abb-75b3bb645314\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.235930 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.235892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" Jul 27 16:57:50.238189 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238167 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-registry-tls\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238296 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eff295e-1fad-4966-9d70-769c896d0561-ca-trust-extracted\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238296 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238244 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-bound-sa-token\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238296 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eff295e-1fad-4966-9d70-769c896d0561-installation-pull-secrets\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238471 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eff295e-1fad-4966-9d70-769c896d0561-trusted-ca\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238471 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238350 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eff295e-1fad-4966-9d70-769c896d0561-registry-certificates\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238471 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238428 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eff295e-1fad-4966-9d70-769c896d0561-image-registry-private-configuration\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238471 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5cqh\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-kube-api-access-v5cqh\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.238755 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.238723 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7eff295e-1fad-4966-9d70-769c896d0561-ca-trust-extracted\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.239351 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.239323 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7eff295e-1fad-4966-9d70-769c896d0561-registry-certificates\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.239475 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.239361 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7eff295e-1fad-4966-9d70-769c896d0561-trusted-ca\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.240790 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.240768 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-registry-tls\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.240899 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.240878 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7eff295e-1fad-4966-9d70-769c896d0561-image-registry-private-configuration\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.240944 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.240889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7eff295e-1fad-4966-9d70-769c896d0561-installation-pull-secrets\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.247719 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.247685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-bound-sa-token\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.248100 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.248080 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5cqh\" (UniqueName: \"kubernetes.io/projected/7eff295e-1fad-4966-9d70-769c896d0561-kube-api-access-v5cqh\") pod \"image-registry-5b8c8d4fdc-xhswd\" (UID: \"7eff295e-1fad-4966-9d70-769c896d0561\") " pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.329334 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.329304 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:50.386840 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.386811 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dnr46"] Jul 27 16:57:50.389934 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:50.389907 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb439a8_fda2_463a_8abb_75b3bb645314.slice/crio-ba000758fcc779b11a469d0c9558fcfd017b3815429c9285aa20ae2757478c82 WatchSource:0}: Error finding container ba000758fcc779b11a469d0c9558fcfd017b3815429c9285aa20ae2757478c82: Status 404 returned error can't find the container with id ba000758fcc779b11a469d0c9558fcfd017b3815429c9285aa20ae2757478c82 Jul 27 16:57:50.493924 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.493826 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8c8d4fdc-xhswd"] Jul 27 16:57:50.498314 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:50.498254 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eff295e_1fad_4966_9d70_769c896d0561.slice/crio-92266ae55a379563223fe8e97e3173776955719a7f8de5cb3e912ff5c2f317db WatchSource:0}: Error finding container 92266ae55a379563223fe8e97e3173776955719a7f8de5cb3e912ff5c2f317db: Status 404 returned error can't find the container with id 92266ae55a379563223fe8e97e3173776955719a7f8de5cb3e912ff5c2f317db Jul 27 16:57:50.530015 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.529987 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" event={"ID":"7eff295e-1fad-4966-9d70-769c896d0561","Type":"ContainerStarted","Data":"92266ae55a379563223fe8e97e3173776955719a7f8de5cb3e912ff5c2f317db"} Jul 27 16:57:50.531273 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:50.531240 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" event={"ID":"3cb439a8-fda2-463a-8abb-75b3bb645314","Type":"ContainerStarted","Data":"ba000758fcc779b11a469d0c9558fcfd017b3815429c9285aa20ae2757478c82"} Jul 27 16:57:51.536748 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:51.536672 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" event={"ID":"7eff295e-1fad-4966-9d70-769c896d0561","Type":"ContainerStarted","Data":"5c6136dd82f430a8911b38e0655810c2739838b7d4152596ae5ddf9e0d97b32a"} Jul 27 16:57:51.573212 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:51.573147 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" podStartSLOduration=2.573127092 podStartE2EDuration="2.573127092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:51.571421067 +0000 UTC m=+77.855580348" watchObservedRunningTime="2026-07-27 16:57:51.573127092 +0000 UTC m=+77.857286378" Jul 27 16:57:52.541525 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:52.541490 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" event={"ID":"3cb439a8-fda2-463a-8abb-75b3bb645314","Type":"ContainerStarted","Data":"78f4909adfe4ab5df9724736688822dbd5f6e6b4c29604c837a9e44c9c4321ee"} Jul 27 16:57:52.541840 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:52.541630 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:57:53.565917 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:53.565867 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dnr46" podStartSLOduration=2.534634572 podStartE2EDuration="4.56585008s" totalImagesPullingTime="2.031215508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:57:53.565151511 +0000 UTC m=+79.849310795" watchObservedRunningTime="2026-07-27 16:57:53.56585008 +0000 UTC m=+79.850009380" Jul 27 16:57:54.473445 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:54.473396 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:54.475775 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:54.475756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e90fe999-65f1-4595-96fb-baeceab626ba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8w66f\" (UID: \"e90fe999-65f1-4595-96fb-baeceab626ba\") " pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:54.507804 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:54.507776 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-567dl" Jul 27 16:57:54.743144 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:54.743065 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bzm7g\"" Jul 27 16:57:54.750827 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:54.750806 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8w66f" Jul 27 16:57:54.904332 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:54.904305 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8w66f"] Jul 27 16:57:54.906577 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:57:54.906549 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode90fe999_65f1_4595_96fb_baeceab626ba.slice/crio-90d31b01c209175cb4ef43ab34ba150f71a5a68ab5b6eb3671f35ab88ce0bd1f WatchSource:0}: Error finding container 90d31b01c209175cb4ef43ab34ba150f71a5a68ab5b6eb3671f35ab88ce0bd1f: Status 404 returned error can't find the container with id 90d31b01c209175cb4ef43ab34ba150f71a5a68ab5b6eb3671f35ab88ce0bd1f Jul 27 16:57:55.556940 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:55.556900 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w66f" event={"ID":"e90fe999-65f1-4595-96fb-baeceab626ba","Type":"ContainerStarted","Data":"90d31b01c209175cb4ef43ab34ba150f71a5a68ab5b6eb3671f35ab88ce0bd1f"} Jul 27 16:57:59.610830 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.610697 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj"] Jul 27 16:57:59.614500 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.614471 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.616862 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.616765 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x"] Jul 27 16:57:59.618311 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.618123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 27 16:57:59.618884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.618859 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 27 16:57:59.618884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.618879 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 27 16:57:59.619772 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.619751 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 27 16:57:59.619870 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.619804 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jqh9v\"" Jul 27 16:57:59.620018 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.620001 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 27 16:57:59.620825 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.620806 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.623507 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.623484 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 27 16:57:59.623622 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.623544 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 27 16:57:59.623890 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.623868 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-67tm5\"" Jul 27 16:57:59.623984 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.623946 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 27 16:57:59.629681 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.629656 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj"] Jul 27 16:57:59.640538 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.640463 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sq2xq"] Jul 27 16:57:59.644256 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.644230 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x"] Jul 27 16:57:59.644397 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.644329 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.646924 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.646905 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 27 16:57:59.647187 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.647169 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 27 16:57:59.647340 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.647304 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qbmgp\"" Jul 27 16:57:59.647516 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.647320 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 27 16:57:59.717729 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717697 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.717729 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717731 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-tls\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717752 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c289849-4cb0-4cc7-8ccf-93097d7f1946-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717771 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq9p6\" (UniqueName: \"kubernetes.io/projected/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-api-access-cq9p6\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60a05bc9-3b07-4582-955a-0b0bc99befa8-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tx99\" (UniqueName: \"kubernetes.io/projected/4c289849-4cb0-4cc7-8ccf-93097d7f1946-kube-api-access-9tx99\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.717954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717949 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c289849-4cb0-4cc7-8ccf-93097d7f1946-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717964 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-textfile\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.717981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbeea8c0-3870-4e06-aba6-e41473363130-metrics-client-ca\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-root\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718048 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/60a05bc9-3b07-4582-955a-0b0bc99befa8-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718073 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-accelerators-collector-config\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718091 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8bs6\" (UniqueName: \"kubernetes.io/projected/fbeea8c0-3870-4e06-aba6-e41473363130-kube-api-access-w8bs6\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718111 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c289849-4cb0-4cc7-8ccf-93097d7f1946-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718132 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-sys\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718147 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-wtmp\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.718183 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.718162 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.818772 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.818738 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60a05bc9-3b07-4582-955a-0b0bc99befa8-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.818964 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.818779 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.818964 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.818823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tx99\" (UniqueName: \"kubernetes.io/projected/4c289849-4cb0-4cc7-8ccf-93097d7f1946-kube-api-access-9tx99\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.818964 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.818845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.818964 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.818872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c289849-4cb0-4cc7-8ccf-93097d7f1946-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.819168 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:59.818992 2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 27 16:57:59.819168 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:59.819065 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-tls podName:60a05bc9-3b07-4582-955a-0b0bc99befa8 nodeName:}" failed. No retries permitted until 2026-07-27 16:58:00.319044396 +0000 UTC m=+86.603203661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-4f75x" (UID: "60a05bc9-3b07-4582-955a-0b0bc99befa8") : secret "kube-state-metrics-tls" not found Jul 27 16:57:59.819727 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.819692 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60a05bc9-3b07-4582-955a-0b0bc99befa8-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.819868 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.819225 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-textfile\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.819868 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.819859 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-textfile\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.819982 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.819861 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbeea8c0-3870-4e06-aba6-e41473363130-metrics-client-ca\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.819982 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.819927 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c289849-4cb0-4cc7-8ccf-93097d7f1946-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.820094 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.819865 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.820094 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-root\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820094 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/60a05bc9-3b07-4582-955a-0b0bc99befa8-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.820094 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820062 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-root\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820103 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-accelerators-collector-config\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8bs6\" (UniqueName: \"kubernetes.io/projected/fbeea8c0-3870-4e06-aba6-e41473363130-kube-api-access-w8bs6\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c289849-4cb0-4cc7-8ccf-93097d7f1946-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820192 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-sys\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-wtmp\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820243 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.820288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820281 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-tls\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820330 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c289849-4cb0-4cc7-8ccf-93097d7f1946-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820354 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbeea8c0-3870-4e06-aba6-e41473363130-metrics-client-ca\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820361 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/60a05bc9-3b07-4582-955a-0b0bc99befa8-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820394 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-wtmp\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820359 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq9p6\" (UniqueName: \"kubernetes.io/projected/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-api-access-cq9p6\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:59.820462 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 27 16:57:59.820685 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:57:59.820509 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-tls podName:fbeea8c0-3870-4e06-aba6-e41473363130 nodeName:}" failed. No retries permitted until 2026-07-27 16:58:00.320496449 +0000 UTC m=+86.604655710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-tls") pod "node-exporter-sq2xq" (UID: "fbeea8c0-3870-4e06-aba6-e41473363130") : secret "node-exporter-tls" not found Jul 27 16:57:59.821089 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820702 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbeea8c0-3870-4e06-aba6-e41473363130-sys\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.821089 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.820877 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-accelerators-collector-config\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.823039 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.822993 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.823136 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.823057 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c289849-4cb0-4cc7-8ccf-93097d7f1946-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.823257 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.823240 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c289849-4cb0-4cc7-8ccf-93097d7f1946-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.823690 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.823671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.827710 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.827687 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tx99\" (UniqueName: \"kubernetes.io/projected/4c289849-4cb0-4cc7-8ccf-93097d7f1946-kube-api-access-9tx99\") pod \"openshift-state-metrics-b8f49f5c6-wk8vj\" (UID: \"4c289849-4cb0-4cc7-8ccf-93097d7f1946\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:57:59.833229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.833205 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq9p6\" (UniqueName: \"kubernetes.io/projected/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-api-access-cq9p6\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:57:59.833927 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.833878 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8bs6\" (UniqueName: \"kubernetes.io/projected/fbeea8c0-3870-4e06-aba6-e41473363130-kube-api-access-w8bs6\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:57:59.929727 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:57:59.929639 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" Jul 27 16:58:00.064327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.064253 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj"] Jul 27 16:58:00.066745 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:00.066718 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c289849_4cb0_4cc7_8ccf_93097d7f1946.slice/crio-7ecba3b08c2cfa58c3492d6e365c6897eba402704a71247b5e85867be6b9a49c WatchSource:0}: Error finding container 7ecba3b08c2cfa58c3492d6e365c6897eba402704a71247b5e85867be6b9a49c: Status 404 returned error can't find the container with id 7ecba3b08c2cfa58c3492d6e365c6897eba402704a71247b5e85867be6b9a49c Jul 27 16:58:00.325692 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.325656 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-tls\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:58:00.325871 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.325747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:58:00.328837 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.328783 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60a05bc9-3b07-4582-955a-0b0bc99befa8-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-4f75x\" (UID: \"60a05bc9-3b07-4582-955a-0b0bc99befa8\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:58:00.330074 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.330026 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbeea8c0-3870-4e06-aba6-e41473363130-node-exporter-tls\") pod \"node-exporter-sq2xq\" (UID: \"fbeea8c0-3870-4e06-aba6-e41473363130\") " pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:58:00.542313 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.542231 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" Jul 27 16:58:00.556049 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.556020 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sq2xq" Jul 27 16:58:00.571302 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:00.571265 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbeea8c0_3870_4e06_aba6_e41473363130.slice/crio-a74f6df2078fdd50cc12342eac19c77d65828e57b098dc9a591d56e9db62b81f WatchSource:0}: Error finding container a74f6df2078fdd50cc12342eac19c77d65828e57b098dc9a591d56e9db62b81f: Status 404 returned error can't find the container with id a74f6df2078fdd50cc12342eac19c77d65828e57b098dc9a591d56e9db62b81f Jul 27 16:58:00.582504 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.582473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w66f" event={"ID":"e90fe999-65f1-4595-96fb-baeceab626ba","Type":"ContainerStarted","Data":"a1b93d431a64cf74a2152dfa44dbef561671e9d231863f18174f5691fe751df7"} Jul 27 16:58:00.584170 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.584119 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq2xq" event={"ID":"fbeea8c0-3870-4e06-aba6-e41473363130","Type":"ContainerStarted","Data":"a74f6df2078fdd50cc12342eac19c77d65828e57b098dc9a591d56e9db62b81f"} Jul 27 16:58:00.586164 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.586139 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" event={"ID":"4c289849-4cb0-4cc7-8ccf-93097d7f1946","Type":"ContainerStarted","Data":"e9c78d37698c19f47c30c3678e8dfa02c9d3bd426aefd43c82961295f88a296f"} Jul 27 16:58:00.586270 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.586194 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" event={"ID":"4c289849-4cb0-4cc7-8ccf-93097d7f1946","Type":"ContainerStarted","Data":"a56fcd46be2893655bac77322743aedca53a92c1541eb0d5a769583c58de2b8c"} Jul 27 16:58:00.586270 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.586226 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" event={"ID":"4c289849-4cb0-4cc7-8ccf-93097d7f1946","Type":"ContainerStarted","Data":"7ecba3b08c2cfa58c3492d6e365c6897eba402704a71247b5e85867be6b9a49c"} Jul 27 16:58:00.651780 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.651218 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:58:00.658850 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.658823 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.662250 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.662224 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 27 16:58:00.663253 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.663231 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rtz5f\"" Jul 27 16:58:00.663545 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.663527 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 27 16:58:00.663739 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.663724 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 27 16:58:00.663916 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.663899 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 27 16:58:00.664205 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.663989 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 27 16:58:00.664205 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.664089 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 27 16:58:00.664355 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.664248 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 27 16:58:00.664449 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.664437 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 27 16:58:00.665732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.665712 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 27 16:58:00.670413 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.670271 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:58:00.714394 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.714058 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x"] Jul 27 16:58:00.716777 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:00.716748 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a05bc9_3b07_4582_955a_0b0bc99befa8.slice/crio-18d4463e84d02a06e3df7d796e612a524f1a59995e7140241d457b76dca63721 WatchSource:0}: Error finding container 18d4463e84d02a06e3df7d796e612a524f1a59995e7140241d457b76dca63721: Status 404 returned error can't find the container with id 18d4463e84d02a06e3df7d796e612a524f1a59995e7140241d457b76dca63721 Jul 27 16:58:00.829439 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829354 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829439 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829408 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-web-config\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829439 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829436 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829691 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829522 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829691 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829558 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829691 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829578 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829691 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829612 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxh7v\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-kube-api-access-jxh7v\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829900 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829900 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829900 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829766 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-out\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829900 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829900 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829827 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-volume\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.829900 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.829883 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.930985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.930945 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931001 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-volume\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931125 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-web-config\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931166 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931190 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931219 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxh7v\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-kube-api-access-jxh7v\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931335 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.931450 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.931357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-out\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.932117 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.932040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.933532 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.933463 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.935620 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.934541 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.935620 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.934888 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.935620 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.935039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.936002 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.935974 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.937724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.936594 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-volume\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.937724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.936756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.937724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.936785 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.937724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.936936 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-out\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.937724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.937031 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.940210 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.940160 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-web-config\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.941452 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.941430 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxh7v\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-kube-api-access-jxh7v\") pod \"alertmanager-main-0\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:00.976286 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:00.976257 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:58:01.157611 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:01.157580 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:58:01.160108 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:01.160077 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06f7932_49bb_4e76_a6b8_ce4958a25b86.slice/crio-22cecb6da2070277c0c5df7c30fd7f4776d43416a46ef86cf80ed9c059255151 WatchSource:0}: Error finding container 22cecb6da2070277c0c5df7c30fd7f4776d43416a46ef86cf80ed9c059255151: Status 404 returned error can't find the container with id 22cecb6da2070277c0c5df7c30fd7f4776d43416a46ef86cf80ed9c059255151 Jul 27 16:58:01.591440 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:01.591403 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" event={"ID":"60a05bc9-3b07-4582-955a-0b0bc99befa8","Type":"ContainerStarted","Data":"18d4463e84d02a06e3df7d796e612a524f1a59995e7140241d457b76dca63721"} Jul 27 16:58:01.592724 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:01.592694 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"22cecb6da2070277c0c5df7c30fd7f4776d43416a46ef86cf80ed9c059255151"} Jul 27 16:58:03.948481 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.948447 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6595d7555b-wsmnr"] Jul 27 16:58:03.952107 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.952081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:03.956774 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.956748 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 27 16:58:03.957008 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.956991 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 27 16:58:03.957083 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.957012 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bng0jemegpqq6\"" Jul 27 16:58:03.957141 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.957019 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 27 16:58:03.957250 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.957233 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pstcb\"" Jul 27 16:58:03.957410 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.957361 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 27 16:58:03.967357 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:03.967238 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6595d7555b-wsmnr"] Jul 27 16:58:04.059134 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059047 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-secret-metrics-server-client-certs\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.059134 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059098 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-secret-metrics-server-tls\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.059321 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhlt\" (UniqueName: \"kubernetes.io/projected/7221b0b2-63bd-4318-94db-d343486615fa-kube-api-access-6hhlt\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.059321 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7221b0b2-63bd-4318-94db-d343486615fa-audit-log\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.059321 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7221b0b2-63bd-4318-94db-d343486615fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.059321 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059305 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7221b0b2-63bd-4318-94db-d343486615fa-metrics-server-audit-profiles\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.059526 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.059327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-client-ca-bundle\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.159746 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhlt\" (UniqueName: \"kubernetes.io/projected/7221b0b2-63bd-4318-94db-d343486615fa-kube-api-access-6hhlt\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.159896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159766 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7221b0b2-63bd-4318-94db-d343486615fa-audit-log\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.159896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7221b0b2-63bd-4318-94db-d343486615fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.159896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7221b0b2-63bd-4318-94db-d343486615fa-metrics-server-audit-profiles\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.159896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-client-ca-bundle\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.159896 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-secret-metrics-server-client-certs\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.160069 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.159915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-secret-metrics-server-tls\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.160732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.160678 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7221b0b2-63bd-4318-94db-d343486615fa-audit-log\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.161006 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.160979 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7221b0b2-63bd-4318-94db-d343486615fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.161271 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.161249 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7221b0b2-63bd-4318-94db-d343486615fa-metrics-server-audit-profiles\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.163309 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.163258 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-secret-metrics-server-tls\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.163431 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.163308 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-client-ca-bundle\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.163431 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.163335 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7221b0b2-63bd-4318-94db-d343486615fa-secret-metrics-server-client-certs\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.170668 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.170500 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhlt\" (UniqueName: \"kubernetes.io/projected/7221b0b2-63bd-4318-94db-d343486615fa-kube-api-access-6hhlt\") pod \"metrics-server-6595d7555b-wsmnr\" (UID: \"7221b0b2-63bd-4318-94db-d343486615fa\") " pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.263752 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.263506 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:04.290536 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.290497 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6"] Jul 27 16:58:04.294141 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.294114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.297437 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.297410 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 27 16:58:04.297437 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.297429 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bszd8\"" Jul 27 16:58:04.303288 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.303241 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6"] Jul 27 16:58:04.461307 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.461277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9241cf42-09bc-4048-8525-fb5811686605-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mrpm6\" (UID: \"9241cf42-09bc-4048-8525-fb5811686605\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.461457 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.461428 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkmg7\" (UniqueName: \"kubernetes.io/projected/9241cf42-09bc-4048-8525-fb5811686605-kube-api-access-lkmg7\") pod \"monitoring-plugin-68b7fc9574-mrpm6\" (UID: \"9241cf42-09bc-4048-8525-fb5811686605\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.562167 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.561853 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9241cf42-09bc-4048-8525-fb5811686605-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mrpm6\" (UID: \"9241cf42-09bc-4048-8525-fb5811686605\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.562167 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.561981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkmg7\" (UniqueName: \"kubernetes.io/projected/9241cf42-09bc-4048-8525-fb5811686605-kube-api-access-lkmg7\") pod \"monitoring-plugin-68b7fc9574-mrpm6\" (UID: \"9241cf42-09bc-4048-8525-fb5811686605\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.565249 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.565205 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9241cf42-09bc-4048-8525-fb5811686605-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-mrpm6\" (UID: \"9241cf42-09bc-4048-8525-fb5811686605\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.571740 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.571717 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkmg7\" (UniqueName: \"kubernetes.io/projected/9241cf42-09bc-4048-8525-fb5811686605-kube-api-access-lkmg7\") pod \"monitoring-plugin-68b7fc9574-mrpm6\" (UID: \"9241cf42-09bc-4048-8525-fb5811686605\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.606017 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.605984 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" event={"ID":"60a05bc9-3b07-4582-955a-0b0bc99befa8","Type":"ContainerStarted","Data":"738859588a71474adbc8ba5174a97edfbaec7eb9b777eb630f8ca37b74d58107"} Jul 27 16:58:04.606703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.606579 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:04.608251 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.608192 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad"} Jul 27 16:58:04.611481 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.611452 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" event={"ID":"4c289849-4cb0-4cc7-8ccf-93097d7f1946","Type":"ContainerStarted","Data":"20eb85ed9b0c036a1a24ecbb9fc3080af6f60d0986a6bd46297fa0b2b60fc02c"} Jul 27 16:58:04.615131 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.615083 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w66f" event={"ID":"e90fe999-65f1-4595-96fb-baeceab626ba","Type":"ContainerStarted","Data":"89214fa6a9aaab26ffd784e0f967b1834461b973de329fa1da4b411a492a76bc"} Jul 27 16:58:04.617569 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.617495 2582 generic.go:356] "Generic (PLEG): container finished" podID="fbeea8c0-3870-4e06-aba6-e41473363130" containerID="a3034daa4c5f2d6e9f61ff823c78c0c3a1070bb2fd3ca07e52763425f6bceb0c" exitCode=0 Jul 27 16:58:04.617569 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.617534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq2xq" event={"ID":"fbeea8c0-3870-4e06-aba6-e41473363130","Type":"ContainerDied","Data":"a3034daa4c5f2d6e9f61ff823c78c0c3a1070bb2fd3ca07e52763425f6bceb0c"} Jul 27 16:58:04.648111 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.647859 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6595d7555b-wsmnr"] Jul 27 16:58:04.669770 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.669693 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-wk8vj" podStartSLOduration=2.065716311 podStartE2EDuration="5.669672444s" totalImagesPullingTime="3.603956133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:04.668433029 +0000 UTC m=+90.952592312" watchObservedRunningTime="2026-07-27 16:58:04.669672444 +0000 UTC m=+90.953831730" Jul 27 16:58:04.782696 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:04.782670 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6"] Jul 27 16:58:04.785602 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:04.785566 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9241cf42_09bc_4048_8525_fb5811686605.slice/crio-76752815eda7268c0f6922a899b64129ab2a1b13a392561d68220afb4ce4e2b1 WatchSource:0}: Error finding container 76752815eda7268c0f6922a899b64129ab2a1b13a392561d68220afb4ce4e2b1: Status 404 returned error can't find the container with id 76752815eda7268c0f6922a899b64129ab2a1b13a392561d68220afb4ce4e2b1 Jul 27 16:58:05.624020 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.623978 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" event={"ID":"9241cf42-09bc-4048-8525-fb5811686605","Type":"ContainerStarted","Data":"76752815eda7268c0f6922a899b64129ab2a1b13a392561d68220afb4ce4e2b1"} Jul 27 16:58:05.627621 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.626962 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq2xq" event={"ID":"fbeea8c0-3870-4e06-aba6-e41473363130","Type":"ContainerStarted","Data":"17c0bd2162801723534fdbec565ceeee4a330d70150b265e9ddd4ce34e340cda"} Jul 27 16:58:05.627621 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.627023 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sq2xq" event={"ID":"fbeea8c0-3870-4e06-aba6-e41473363130","Type":"ContainerStarted","Data":"254cca92cf13c3a247a33ee91c6695dded6a6c4e2875d30ec541c441019d2de8"} Jul 27 16:58:05.629941 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.629800 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" event={"ID":"60a05bc9-3b07-4582-955a-0b0bc99befa8","Type":"ContainerStarted","Data":"af30acaf13d1f0a2de008c68d99ddc0c829f6f842b2469eae3a3c7b8c151780e"} Jul 27 16:58:05.629941 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.629863 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" event={"ID":"60a05bc9-3b07-4582-955a-0b0bc99befa8","Type":"ContainerStarted","Data":"a67680e98a620dc056242e66524dcdf1b408bb9e58c0c6c6ed5a7a3593033db8"} Jul 27 16:58:05.632564 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.632535 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad" exitCode=0 Jul 27 16:58:05.632686 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.632623 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad"} Jul 27 16:58:05.635806 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.635753 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" event={"ID":"7221b0b2-63bd-4318-94db-d343486615fa","Type":"ContainerStarted","Data":"77c58ea5638cb7fa3a298d2bbb5124af4c6ffa285050859cb473041b0f4ef110"} Jul 27 16:58:05.681915 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.681853 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sq2xq" podStartSLOduration=3.395435749 podStartE2EDuration="6.681832059s" totalImagesPullingTime="3.210303455s" totalInitContainerRuntime="76.092855ms" isStatefulPod=true podCreationTimestamp="2026-07-27 16:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:05.680456455 +0000 UTC m=+91.964615740" watchObservedRunningTime="2026-07-27 16:58:05.681832059 +0000 UTC m=+91.965991343" Jul 27 16:58:05.702807 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:05.702747 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-4f75x" podStartSLOduration=2.936708075 podStartE2EDuration="6.702723501s" totalImagesPullingTime="3.766015426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:05.702334247 +0000 UTC m=+91.986493531" watchObservedRunningTime="2026-07-27 16:58:05.702723501 +0000 UTC m=+91.986882786" Jul 27 16:58:09.657936 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.657894 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2"} Jul 27 16:58:09.658354 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.658003 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda"} Jul 27 16:58:09.658354 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.658050 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711"} Jul 27 16:58:09.658354 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.658095 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f"} Jul 27 16:58:09.658354 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.658131 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34"} Jul 27 16:58:09.659441 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.659415 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" event={"ID":"7221b0b2-63bd-4318-94db-d343486615fa","Type":"ContainerStarted","Data":"30f8f353a49c426b2a16c16fcd97e2729227c7338ace0344fea3b16bc7f4d197"} Jul 27 16:58:09.661297 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.661270 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8w66f" event={"ID":"e90fe999-65f1-4595-96fb-baeceab626ba","Type":"ContainerStarted","Data":"b9082dc1eeb28967b2aeb1733cfe9e8bc8d74077b02e6943b47c73617567ad0c"} Jul 27 16:58:09.662594 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.662576 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" event={"ID":"9241cf42-09bc-4048-8525-fb5811686605","Type":"ContainerStarted","Data":"55ccb93cd2fdaaa64cc3c6addb434acff55cb51a23b2241323255b3957a8d939"} Jul 27 16:58:09.662686 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.662671 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:09.667414 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.667372 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" Jul 27 16:58:09.680883 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.680807 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" podStartSLOduration=2.529317927 podStartE2EDuration="6.680793589s" totalImagesPullingTime="4.151475662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:09.679543906 +0000 UTC m=+95.963703228" watchObservedRunningTime="2026-07-27 16:58:09.680793589 +0000 UTC m=+95.964952872" Jul 27 16:58:09.698387 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.698330 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-mrpm6" podStartSLOduration=1.680238691 podStartE2EDuration="5.698314955s" totalImagesPullingTime="4.018076264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:09.696341563 +0000 UTC m=+95.980500845" watchObservedRunningTime="2026-07-27 16:58:09.698314955 +0000 UTC m=+95.982474237" Jul 27 16:58:09.720095 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:09.720044 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8w66f" podStartSLOduration=33.935090563 podStartE2EDuration="47.720028372s" totalImagesPullingTime="13.784937809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 16:57:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:09.718851704 +0000 UTC m=+96.003010986" watchObservedRunningTime="2026-07-27 16:58:09.720028372 +0000 UTC m=+96.004187654" Jul 27 16:58:11.489854 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:11.489821 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rhgrq" Jul 27 16:58:11.673224 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:11.673184 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerStarted","Data":"1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff"} Jul 27 16:58:11.716334 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:11.716274 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.997108407 podStartE2EDuration="11.71625687s" totalImagesPullingTime="9.691508014s" totalInitContainerRuntime="27.640449ms" isStatefulPod=false podCreationTimestamp="2026-07-27 16:58:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:11.712598705 +0000 UTC m=+97.996757988" watchObservedRunningTime="2026-07-27 16:58:11.71625687 +0000 UTC m=+98.000416191" Jul 27 16:58:13.549519 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:13.549489 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b8c8d4fdc-xhswd" Jul 27 16:58:14.629278 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.629242 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-47rzq"] Jul 27 16:58:14.634830 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.634805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:14.638047 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.638024 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 27 16:58:14.638538 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.638522 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 27 16:58:14.639749 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.639733 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-x642j\"" Jul 27 16:58:14.644861 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.644840 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-47rzq"] Jul 27 16:58:14.657134 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.657111 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlmzb\" (UniqueName: \"kubernetes.io/projected/055c4bbc-0d60-4538-b1d4-9f54225ff3a2-kube-api-access-hlmzb\") pod \"downloads-7c97dc4555-47rzq\" (UID: \"055c4bbc-0d60-4538-b1d4-9f54225ff3a2\") " pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:14.757491 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.757462 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlmzb\" (UniqueName: \"kubernetes.io/projected/055c4bbc-0d60-4538-b1d4-9f54225ff3a2-kube-api-access-hlmzb\") pod \"downloads-7c97dc4555-47rzq\" (UID: \"055c4bbc-0d60-4538-b1d4-9f54225ff3a2\") " pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:14.767686 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.767659 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlmzb\" (UniqueName: \"kubernetes.io/projected/055c4bbc-0d60-4538-b1d4-9f54225ff3a2-kube-api-access-hlmzb\") pod \"downloads-7c97dc4555-47rzq\" (UID: \"055c4bbc-0d60-4538-b1d4-9f54225ff3a2\") " pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:14.944141 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:14.944060 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:15.096320 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:15.096292 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-47rzq"] Jul 27 16:58:15.097531 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:15.097508 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055c4bbc_0d60_4538_b1d4_9f54225ff3a2.slice/crio-62d1ac9d3b53d44048e40a98f040f738f4083751345214132c3ced1fb6cc9cdc WatchSource:0}: Error finding container 62d1ac9d3b53d44048e40a98f040f738f4083751345214132c3ced1fb6cc9cdc: Status 404 returned error can't find the container with id 62d1ac9d3b53d44048e40a98f040f738f4083751345214132c3ced1fb6cc9cdc Jul 27 16:58:15.692744 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:15.692706 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-47rzq" event={"ID":"055c4bbc-0d60-4538-b1d4-9f54225ff3a2","Type":"ContainerStarted","Data":"62d1ac9d3b53d44048e40a98f040f738f4083751345214132c3ced1fb6cc9cdc"} Jul 27 16:58:23.740768 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.740729 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c9f6666c7-nk4nv"] Jul 27 16:58:23.748839 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.748810 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.751874 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.751844 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 27 16:58:23.753868 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.753751 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 27 16:58:23.753868 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.753793 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 27 16:58:23.754060 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.754044 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zk7th\"" Jul 27 16:58:23.754155 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.754129 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 27 16:58:23.754710 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.754691 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 27 16:58:23.756195 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.756136 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9f6666c7-nk4nv"] Jul 27 16:58:23.841602 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.841566 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-serving-cert\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.841783 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.841624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-oauth-config\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.841783 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.841701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsbsg\" (UniqueName: \"kubernetes.io/projected/0049617b-6725-46cf-a2d0-a02a59eb0751-kube-api-access-fsbsg\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.841783 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.841748 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-service-ca\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.842008 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.841797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-console-config\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.842008 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.841894 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-oauth-serving-cert\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.942511 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.942468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-serving-cert\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.942719 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.942530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-oauth-config\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.942719 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.942563 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsbsg\" (UniqueName: \"kubernetes.io/projected/0049617b-6725-46cf-a2d0-a02a59eb0751-kube-api-access-fsbsg\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.942719 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.942606 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-service-ca\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.942719 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.942640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-console-config\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.942719 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.942705 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-oauth-serving-cert\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.943747 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.943698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-oauth-serving-cert\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.943747 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.943725 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-console-config\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.943925 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.943901 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-service-ca\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.945251 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.945210 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-oauth-config\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.945426 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.945401 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-serving-cert\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:23.952492 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:23.952467 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsbsg\" (UniqueName: \"kubernetes.io/projected/0049617b-6725-46cf-a2d0-a02a59eb0751-kube-api-access-fsbsg\") pod \"console-5c9f6666c7-nk4nv\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:24.061134 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:24.061029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:24.220874 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:24.220820 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9f6666c7-nk4nv"] Jul 27 16:58:24.224036 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:24.224002 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0049617b_6725_46cf_a2d0_a02a59eb0751.slice/crio-a184b8119b46a03a3f89b6931355a716a0d901d6c3bd1657d6b249fceb0bbbbe WatchSource:0}: Error finding container a184b8119b46a03a3f89b6931355a716a0d901d6c3bd1657d6b249fceb0bbbbe: Status 404 returned error can't find the container with id a184b8119b46a03a3f89b6931355a716a0d901d6c3bd1657d6b249fceb0bbbbe Jul 27 16:58:24.264180 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:24.264148 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:24.264396 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:24.264210 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:24.735782 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:24.735744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9f6666c7-nk4nv" event={"ID":"0049617b-6725-46cf-a2d0-a02a59eb0751","Type":"ContainerStarted","Data":"a184b8119b46a03a3f89b6931355a716a0d901d6c3bd1657d6b249fceb0bbbbe"} Jul 27 16:58:31.767329 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:31.767292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-47rzq" event={"ID":"055c4bbc-0d60-4538-b1d4-9f54225ff3a2","Type":"ContainerStarted","Data":"97ff4ff8e4b4b132b8b9257ab6540e1288d356612d5c8d7e9019f6a95f38998b"} Jul 27 16:58:31.767736 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:31.767495 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:31.769168 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:31.769144 2582 patch_prober.go:28] interesting pod/downloads-7c97dc4555-47rzq container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.22:8080/\": dial tcp 10.133.0.22:8080: connect: connection refused" start-of-body= Jul 27 16:58:31.769258 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:31.769187 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-47rzq" podUID="055c4bbc-0d60-4538-b1d4-9f54225ff3a2" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.22:8080/\": dial tcp 10.133.0.22:8080: connect: connection refused" Jul 27 16:58:31.788327 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:31.788273 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-47rzq" podStartSLOduration=1.187676111 podStartE2EDuration="17.788255209s" totalImagesPullingTime="16.600579098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:31.786347229 +0000 UTC m=+118.070506514" watchObservedRunningTime="2026-07-27 16:58:31.788255209 +0000 UTC m=+118.072414503" Jul 27 16:58:32.123418 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.123356 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d7d7694b5-r2njg"] Jul 27 16:58:32.128309 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.128257 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.137946 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.137745 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 27 16:58:32.139912 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.139845 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d7d7694b5-r2njg"] Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223624 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-service-ca\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223677 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf5h8\" (UniqueName: \"kubernetes.io/projected/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-kube-api-access-jf5h8\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223712 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-serving-cert\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223741 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-config\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-trusted-ca-bundle\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223838 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-oauth-config\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.223954 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.223856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-oauth-serving-cert\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.324856 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.324791 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-serving-cert\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.324856 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.324851 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-config\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.325971 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.324959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-trusted-ca-bundle\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.325971 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.325002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-oauth-config\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.325971 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.325923 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-config\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.326209 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.326176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-oauth-serving-cert\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.326331 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.326245 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-service-ca\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.326331 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.326286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf5h8\" (UniqueName: \"kubernetes.io/projected/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-kube-api-access-jf5h8\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.327344 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.326688 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-trusted-ca-bundle\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.327344 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.326913 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-oauth-serving-cert\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.327344 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.327297 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-service-ca\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.329672 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.329630 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-oauth-config\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.330574 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.330501 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-serving-cert\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.338073 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.338035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf5h8\" (UniqueName: \"kubernetes.io/projected/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-kube-api-access-jf5h8\") pod \"console-6d7d7694b5-r2njg\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.447906 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.447821 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:32.651928 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.651632 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d7d7694b5-r2njg"] Jul 27 16:58:32.655205 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:58:32.655040 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd157d9bc_8e5b_4052_bbcd_ecef49d4a276.slice/crio-0ccddea0b9a7799db05f0c493cecb725da42167a2cdfb992da6aa775cde4eccc WatchSource:0}: Error finding container 0ccddea0b9a7799db05f0c493cecb725da42167a2cdfb992da6aa775cde4eccc: Status 404 returned error can't find the container with id 0ccddea0b9a7799db05f0c493cecb725da42167a2cdfb992da6aa775cde4eccc Jul 27 16:58:32.775580 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.775446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7d7694b5-r2njg" event={"ID":"d157d9bc-8e5b-4052-bbcd-ecef49d4a276","Type":"ContainerStarted","Data":"0ccddea0b9a7799db05f0c493cecb725da42167a2cdfb992da6aa775cde4eccc"} Jul 27 16:58:32.789899 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:32.789762 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-47rzq" Jul 27 16:58:36.799608 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:36.799569 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9f6666c7-nk4nv" event={"ID":"0049617b-6725-46cf-a2d0-a02a59eb0751","Type":"ContainerStarted","Data":"191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4"} Jul 27 16:58:36.801443 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:36.801413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7d7694b5-r2njg" event={"ID":"d157d9bc-8e5b-4052-bbcd-ecef49d4a276","Type":"ContainerStarted","Data":"07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57"} Jul 27 16:58:36.820436 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:36.820367 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c9f6666c7-nk4nv" podStartSLOduration=2.430369091 podStartE2EDuration="13.820350233s" totalImagesPullingTime="11.389981142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:36.819147999 +0000 UTC m=+123.103307317" watchObservedRunningTime="2026-07-27 16:58:36.820350233 +0000 UTC m=+123.104509516" Jul 27 16:58:36.839853 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:36.839799 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d7d7694b5-r2njg" podStartSLOduration=1.757736889 podStartE2EDuration="4.839775202s" totalImagesPullingTime="3.082038313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:58:36.837962713 +0000 UTC m=+123.122121999" watchObservedRunningTime="2026-07-27 16:58:36.839775202 +0000 UTC m=+123.123934486" Jul 27 16:58:37.807894 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:37.807858 2582 generic.go:356] "Generic (PLEG): container finished" podID="c9d28086-fbfc-4d08-9d3d-1db5193d3c1f" containerID="351a5520b4144ea77506d9aa6f97315286467301c6ef4329c21249a88bec8929" exitCode=0 Jul 27 16:58:37.808344 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:37.807946 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bxswn" event={"ID":"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f","Type":"ContainerDied","Data":"351a5520b4144ea77506d9aa6f97315286467301c6ef4329c21249a88bec8929"} Jul 27 16:58:37.808591 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:37.808568 2582 scope.go:117] "RemoveContainer" containerID="351a5520b4144ea77506d9aa6f97315286467301c6ef4329c21249a88bec8929" Jul 27 16:58:38.660626 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:38.660589 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/init-config-reloader/0.log" Jul 27 16:58:38.666889 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:38.666861 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/alertmanager/0.log" Jul 27 16:58:38.755987 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:38.755959 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/config-reloader/0.log" Jul 27 16:58:38.815228 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:38.815189 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-bxswn" event={"ID":"c9d28086-fbfc-4d08-9d3d-1db5193d3c1f","Type":"ContainerStarted","Data":"8585392dca4c103241d63de7c86a4136dc2869d55db8d8ece8aa44fb5aeb30f0"} Jul 27 16:58:38.963033 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:38.962949 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/kube-rbac-proxy-web/0.log" Jul 27 16:58:39.159474 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:39.159403 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/kube-rbac-proxy/0.log" Jul 27 16:58:39.355653 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:39.355625 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/kube-rbac-proxy-metric/0.log" Jul 27 16:58:39.555879 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:39.555848 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d06f7932-49bb-4e76-a6b8-ce4958a25b86/prom-label-proxy/0.log" Jul 27 16:58:39.955906 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:39.955871 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4f75x_60a05bc9-3b07-4582-955a-0b0bc99befa8/kube-state-metrics/0.log" Jul 27 16:58:40.155250 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:40.155212 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4f75x_60a05bc9-3b07-4582-955a-0b0bc99befa8/kube-rbac-proxy-main/0.log" Jul 27 16:58:40.355993 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:40.355959 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4f75x_60a05bc9-3b07-4582-955a-0b0bc99befa8/kube-rbac-proxy-self/0.log" Jul 27 16:58:40.555774 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:40.555738 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6595d7555b-wsmnr_7221b0b2-63bd-4318-94db-d343486615fa/metrics-server/0.log" Jul 27 16:58:40.758236 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:40.758150 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-mrpm6_9241cf42-09bc-4048-8525-fb5811686605/monitoring-plugin/0.log" Jul 27 16:58:41.555866 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:41.555842 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq2xq_fbeea8c0-3870-4e06-aba6-e41473363130/init-textfile/0.log" Jul 27 16:58:41.755732 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:41.755705 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq2xq_fbeea8c0-3870-4e06-aba6-e41473363130/node-exporter/0.log" Jul 27 16:58:41.955515 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:41.955487 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq2xq_fbeea8c0-3870-4e06-aba6-e41473363130/kube-rbac-proxy/0.log" Jul 27 16:58:42.448637 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.448604 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:42.448792 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.448689 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:42.453903 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.453869 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:42.757255 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.757182 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-wk8vj_4c289849-4cb0-4cc7-8ccf-93097d7f1946/kube-rbac-proxy-main/0.log" Jul 27 16:58:42.833420 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.833362 2582 generic.go:356] "Generic (PLEG): container finished" podID="6c84d615-a6b0-488c-9dcd-3fcc11bbea80" containerID="0ff75e3146d4c145afd8fc7320a9d8479d31de0ed0196df87278b0edd9c3f4ff" exitCode=0 Jul 27 16:58:42.833573 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.833436 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" event={"ID":"6c84d615-a6b0-488c-9dcd-3fcc11bbea80","Type":"ContainerDied","Data":"0ff75e3146d4c145afd8fc7320a9d8479d31de0ed0196df87278b0edd9c3f4ff"} Jul 27 16:58:42.833938 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.833910 2582 scope.go:117] "RemoveContainer" containerID="0ff75e3146d4c145afd8fc7320a9d8479d31de0ed0196df87278b0edd9c3f4ff" Jul 27 16:58:42.837507 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.837487 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 16:58:42.903777 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.903741 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9f6666c7-nk4nv"] Jul 27 16:58:42.955195 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:42.955164 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-wk8vj_4c289849-4cb0-4cc7-8ccf-93097d7f1946/kube-rbac-proxy-self/0.log" Jul 27 16:58:43.157409 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:43.157363 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-wk8vj_4c289849-4cb0-4cc7-8ccf-93097d7f1946/openshift-state-metrics/0.log" Jul 27 16:58:43.840469 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:43.840432 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-tmj4l" event={"ID":"6c84d615-a6b0-488c-9dcd-3fcc11bbea80","Type":"ContainerStarted","Data":"240d708d666b23b2da9d750646891b5472c9ca855b65e6fa7b2a9a3b5ddc3df9"} Jul 27 16:58:44.062203 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:44.062167 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:58:44.269569 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:44.269491 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:44.273344 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:44.273312 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6595d7555b-wsmnr" Jul 27 16:58:46.554737 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:46.554689 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-dnr46_3cb439a8-fda2-463a-8abb-75b3bb645314/networking-console-plugin/0.log" Jul 27 16:58:47.157238 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:47.157209 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5c9f6666c7-nk4nv_0049617b-6725-46cf-a2d0-a02a59eb0751/console/0.log" Jul 27 16:58:47.356464 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:47.356436 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d7d7694b5-r2njg_d157d9bc-8e5b-4052-bbcd-ecef49d4a276/console/0.log" Jul 27 16:58:47.556407 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:47.556322 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-47rzq_055c4bbc-0d60-4538-b1d4-9f54225ff3a2/download-server/0.log" Jul 27 16:58:52.881970 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:52.881922 2582 generic.go:356] "Generic (PLEG): container finished" podID="369747a7-45ae-4fb0-ae48-ca89ea345ec7" containerID="f096b96bd4e54d9c1c04561aae9007df5c3ce88fe9d9bc1301dc1a13fca779a2" exitCode=0 Jul 27 16:58:52.882504 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:52.882006 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" event={"ID":"369747a7-45ae-4fb0-ae48-ca89ea345ec7","Type":"ContainerDied","Data":"f096b96bd4e54d9c1c04561aae9007df5c3ce88fe9d9bc1301dc1a13fca779a2"} Jul 27 16:58:52.882504 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:52.882410 2582 scope.go:117] "RemoveContainer" containerID="f096b96bd4e54d9c1c04561aae9007df5c3ce88fe9d9bc1301dc1a13fca779a2" Jul 27 16:58:53.887518 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:58:53.887482 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-l7hs7" event={"ID":"369747a7-45ae-4fb0-ae48-ca89ea345ec7","Type":"ContainerStarted","Data":"06e88460dbf7084e6519b9c0552e3c6d411b222e1ed8cfb22c8b39b51ae21f92"} Jul 27 16:59:07.923320 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:07.923269 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c9f6666c7-nk4nv" podUID="0049617b-6725-46cf-a2d0-a02a59eb0751" containerName="console" containerID="cri-o://191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4" gracePeriod=15 Jul 27 16:59:08.219917 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.219884 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:59:08.253393 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253350 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-console-config\") pod \"0049617b-6725-46cf-a2d0-a02a59eb0751\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " Jul 27 16:59:08.253549 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253420 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-service-ca\") pod \"0049617b-6725-46cf-a2d0-a02a59eb0751\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " Jul 27 16:59:08.253549 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253488 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-serving-cert\") pod \"0049617b-6725-46cf-a2d0-a02a59eb0751\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " Jul 27 16:59:08.253549 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253533 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsbsg\" (UniqueName: \"kubernetes.io/projected/0049617b-6725-46cf-a2d0-a02a59eb0751-kube-api-access-fsbsg\") pod \"0049617b-6725-46cf-a2d0-a02a59eb0751\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " Jul 27 16:59:08.253711 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253556 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-oauth-serving-cert\") pod \"0049617b-6725-46cf-a2d0-a02a59eb0751\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " Jul 27 16:59:08.253711 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253597 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-oauth-config\") pod \"0049617b-6725-46cf-a2d0-a02a59eb0751\" (UID: \"0049617b-6725-46cf-a2d0-a02a59eb0751\") " Jul 27 16:59:08.253814 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253763 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-console-config" (OuterVolumeSpecName: "console-config") pod "0049617b-6725-46cf-a2d0-a02a59eb0751" (UID: "0049617b-6725-46cf-a2d0-a02a59eb0751"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 16:59:08.254265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253885 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-service-ca" (OuterVolumeSpecName: "service-ca") pod "0049617b-6725-46cf-a2d0-a02a59eb0751" (UID: "0049617b-6725-46cf-a2d0-a02a59eb0751"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 16:59:08.254265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.253918 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0049617b-6725-46cf-a2d0-a02a59eb0751" (UID: "0049617b-6725-46cf-a2d0-a02a59eb0751"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 16:59:08.254265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.254008 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-oauth-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:08.254265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.254027 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-console-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:08.254265 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.254040 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0049617b-6725-46cf-a2d0-a02a59eb0751-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:08.255902 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.255874 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0049617b-6725-46cf-a2d0-a02a59eb0751" (UID: "0049617b-6725-46cf-a2d0-a02a59eb0751"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:08.256109 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.256083 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0049617b-6725-46cf-a2d0-a02a59eb0751" (UID: "0049617b-6725-46cf-a2d0-a02a59eb0751"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:08.256109 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.256094 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0049617b-6725-46cf-a2d0-a02a59eb0751-kube-api-access-fsbsg" (OuterVolumeSpecName: "kube-api-access-fsbsg") pod "0049617b-6725-46cf-a2d0-a02a59eb0751" (UID: "0049617b-6725-46cf-a2d0-a02a59eb0751"). InnerVolumeSpecName "kube-api-access-fsbsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 16:59:08.354726 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.354683 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:08.354726 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.354718 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsbsg\" (UniqueName: \"kubernetes.io/projected/0049617b-6725-46cf-a2d0-a02a59eb0751-kube-api-access-fsbsg\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:08.354726 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.354732 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0049617b-6725-46cf-a2d0-a02a59eb0751-console-oauth-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:08.952567 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.952532 2582 generic.go:356] "Generic (PLEG): container finished" podID="0049617b-6725-46cf-a2d0-a02a59eb0751" containerID="191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4" exitCode=0 Jul 27 16:59:08.952993 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.952574 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9f6666c7-nk4nv" event={"ID":"0049617b-6725-46cf-a2d0-a02a59eb0751","Type":"ContainerDied","Data":"191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4"} Jul 27 16:59:08.952993 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.952582 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9f6666c7-nk4nv" Jul 27 16:59:08.952993 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.952632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9f6666c7-nk4nv" event={"ID":"0049617b-6725-46cf-a2d0-a02a59eb0751","Type":"ContainerDied","Data":"a184b8119b46a03a3f89b6931355a716a0d901d6c3bd1657d6b249fceb0bbbbe"} Jul 27 16:59:08.952993 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.952664 2582 scope.go:117] "RemoveContainer" containerID="191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4" Jul 27 16:59:08.971444 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.971422 2582 scope.go:117] "RemoveContainer" containerID="191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4" Jul 27 16:59:08.971757 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:08.971736 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4\": container with ID starting with 191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4 not found: ID does not exist" containerID="191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4" Jul 27 16:59:08.971814 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.971766 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4"} err="failed to get container status \"191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4\": rpc error: code = NotFound desc = could not find container \"191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4\": container with ID starting with 191bc09344e40b4d81213dadc10fec5ed052166238e58e9fcebfeab0b0e648d4 not found: ID does not exist" Jul 27 16:59:08.975311 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.975287 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9f6666c7-nk4nv"] Jul 27 16:59:08.979945 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:08.979914 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c9f6666c7-nk4nv"] Jul 27 16:59:10.197860 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:10.197828 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0049617b-6725-46cf-a2d0-a02a59eb0751" path="/var/lib/kubelet/pods/0049617b-6725-46cf-a2d0-a02a59eb0751/volumes" Jul 27 16:59:19.958113 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958074 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:59:19.958667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958352 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="alertmanager" containerID="cri-o://c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34" gracePeriod=120 Jul 27 16:59:19.958667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958437 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-metric" containerID="cri-o://267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2" gracePeriod=120 Jul 27 16:59:19.958667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958503 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="prom-label-proxy" containerID="cri-o://1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff" gracePeriod=120 Jul 27 16:59:19.958667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958531 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy" containerID="cri-o://57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda" gracePeriod=120 Jul 27 16:59:19.958667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958514 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="config-reloader" containerID="cri-o://9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f" gracePeriod=120 Jul 27 16:59:19.958667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:19.958497 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-web" containerID="cri-o://dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711" gracePeriod=120 Jul 27 16:59:21.004533 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004496 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff" exitCode=0 Jul 27 16:59:21.004533 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004527 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2" exitCode=0 Jul 27 16:59:21.004533 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004536 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda" exitCode=0 Jul 27 16:59:21.004533 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004542 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f" exitCode=0 Jul 27 16:59:21.004985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004550 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34" exitCode=0 Jul 27 16:59:21.004985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff"} Jul 27 16:59:21.004985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004629 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2"} Jul 27 16:59:21.004985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda"} Jul 27 16:59:21.004985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004680 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f"} Jul 27 16:59:21.004985 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.004704 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34"} Jul 27 16:59:21.242887 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.242858 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:21.373837 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373803 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373848 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373879 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-metrics-client-ca\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373912 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-web\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373936 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxh7v\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-kube-api-access-jxh7v\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373960 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-trusted-ca-bundle\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374005 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.373982 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-tls-assets\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374014 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-main-tls\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374039 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-out\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374083 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-main-db\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374140 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-volume\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374168 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-cluster-tls-config\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374308 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374199 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-web-config\") pod \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\" (UID: \"d06f7932-49bb-4e76-a6b8-ce4958a25b86\") " Jul 27 16:59:21.374643 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374355 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 16:59:21.374643 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374363 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 16:59:21.374643 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374547 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.374643 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374568 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d06f7932-49bb-4e76-a6b8-ce4958a25b86-metrics-client-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.375566 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.374905 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 16:59:21.376843 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.376813 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.377174 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.377142 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.377174 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.377165 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-out" (OuterVolumeSpecName: "config-out") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 16:59:21.377487 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.377455 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 16:59:21.377679 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.377652 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-volume" (OuterVolumeSpecName: "config-volume") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.377884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.377854 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-kube-api-access-jxh7v" (OuterVolumeSpecName: "kube-api-access-jxh7v") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "kube-api-access-jxh7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 16:59:21.378335 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.378308 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.378486 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.378395 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.381293 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.381264 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.388219 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.388191 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-web-config" (OuterVolumeSpecName: "web-config") pod "d06f7932-49bb-4e76-a6b8-ce4958a25b86" (UID: "d06f7932-49bb-4e76-a6b8-ce4958a25b86"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 16:59:21.475660 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475623 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475660 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475655 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475660 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475666 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxh7v\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-kube-api-access-jxh7v\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475675 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d06f7932-49bb-4e76-a6b8-ce4958a25b86-tls-assets\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475685 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-main-tls\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475693 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-out\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475701 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d06f7932-49bb-4e76-a6b8-ce4958a25b86-alertmanager-main-db\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475711 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-config-volume\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475719 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-cluster-tls-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475727 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-web-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:21.475884 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:21.475735 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d06f7932-49bb-4e76-a6b8-ce4958a25b86-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 16:59:22.011905 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.011872 2582 generic.go:356] "Generic (PLEG): container finished" podID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerID="dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711" exitCode=0 Jul 27 16:59:22.012277 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.011946 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.012277 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.011958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711"} Jul 27 16:59:22.012277 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.012029 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d06f7932-49bb-4e76-a6b8-ce4958a25b86","Type":"ContainerDied","Data":"22cecb6da2070277c0c5df7c30fd7f4776d43416a46ef86cf80ed9c059255151"} Jul 27 16:59:22.012277 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.012062 2582 scope.go:117] "RemoveContainer" containerID="1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff" Jul 27 16:59:22.022064 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.021990 2582 scope.go:117] "RemoveContainer" containerID="267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2" Jul 27 16:59:22.029693 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.029671 2582 scope.go:117] "RemoveContainer" containerID="57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda" Jul 27 16:59:22.036701 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.036681 2582 scope.go:117] "RemoveContainer" containerID="dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711" Jul 27 16:59:22.039277 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.039259 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:59:22.043885 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.043865 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:59:22.044583 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.044567 2582 scope.go:117] "RemoveContainer" containerID="9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f" Jul 27 16:59:22.051398 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.051363 2582 scope.go:117] "RemoveContainer" containerID="c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34" Jul 27 16:59:22.059293 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.059278 2582 scope.go:117] "RemoveContainer" containerID="91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad" Jul 27 16:59:22.066171 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066155 2582 scope.go:117] "RemoveContainer" containerID="1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff" Jul 27 16:59:22.066431 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.066410 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff\": container with ID starting with 1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff not found: ID does not exist" containerID="1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff" Jul 27 16:59:22.066515 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066439 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff"} err="failed to get container status \"1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff\": rpc error: code = NotFound desc = could not find container \"1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff\": container with ID starting with 1f41043dda6a9857df54d353a194eea27614913da41679c37b572e1beffc58ff not found: ID does not exist" Jul 27 16:59:22.066515 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066457 2582 scope.go:117] "RemoveContainer" containerID="267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2" Jul 27 16:59:22.066687 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.066669 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2\": container with ID starting with 267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2 not found: ID does not exist" containerID="267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2" Jul 27 16:59:22.066728 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066690 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2"} err="failed to get container status \"267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2\": rpc error: code = NotFound desc = could not find container \"267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2\": container with ID starting with 267d3170690baee1c4c7722c3cee3c6bcbd25b5ac1efe75ab93e8babe84fddc2 not found: ID does not exist" Jul 27 16:59:22.066728 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066703 2582 scope.go:117] "RemoveContainer" containerID="57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda" Jul 27 16:59:22.066926 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.066912 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda\": container with ID starting with 57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda not found: ID does not exist" containerID="57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda" Jul 27 16:59:22.066965 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066930 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda"} err="failed to get container status \"57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda\": rpc error: code = NotFound desc = could not find container \"57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda\": container with ID starting with 57ffbe0f7f50b69769b4f38128f8027cc48e51fe238a44d1c54f89c33c99ebda not found: ID does not exist" Jul 27 16:59:22.066965 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.066942 2582 scope.go:117] "RemoveContainer" containerID="dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711" Jul 27 16:59:22.067141 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.067126 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711\": container with ID starting with dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711 not found: ID does not exist" containerID="dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711" Jul 27 16:59:22.067179 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067144 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711"} err="failed to get container status \"dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711\": rpc error: code = NotFound desc = could not find container \"dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711\": container with ID starting with dff14aed177efd7de8fbfb8a1891e019fbd69b45c591b043973e6dede9045711 not found: ID does not exist" Jul 27 16:59:22.067179 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067172 2582 scope.go:117] "RemoveContainer" containerID="9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f" Jul 27 16:59:22.067415 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.067395 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f\": container with ID starting with 9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f not found: ID does not exist" containerID="9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f" Jul 27 16:59:22.067475 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067421 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f"} err="failed to get container status \"9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f\": rpc error: code = NotFound desc = could not find container \"9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f\": container with ID starting with 9be8720d70933d3b7c306ebe1a520f1aee5f278de22b231f7131067367d56d0f not found: ID does not exist" Jul 27 16:59:22.067475 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067436 2582 scope.go:117] "RemoveContainer" containerID="c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34" Jul 27 16:59:22.067635 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.067618 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34\": container with ID starting with c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34 not found: ID does not exist" containerID="c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34" Jul 27 16:59:22.067669 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067638 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34"} err="failed to get container status \"c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34\": rpc error: code = NotFound desc = could not find container \"c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34\": container with ID starting with c1700601caa30e8f1d3dc9121de8a9741ef9f810c1b2b3b22f064688311c3f34 not found: ID does not exist" Jul 27 16:59:22.067669 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067651 2582 scope.go:117] "RemoveContainer" containerID="91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad" Jul 27 16:59:22.067857 ip-10-0-135-54 kubenswrapper[2582]: E0727 16:59:22.067840 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad\": container with ID starting with 91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad not found: ID does not exist" containerID="91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad" Jul 27 16:59:22.067904 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.067860 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad"} err="failed to get container status \"91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad\": rpc error: code = NotFound desc = could not find container \"91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad\": container with ID starting with 91e8fb5a49e3856b155ac86216be3363f839ffa555c1bb6777ac74bd555132ad not found: ID does not exist" Jul 27 16:59:22.072035 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072016 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:59:22.072298 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072287 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-web" Jul 27 16:59:22.072338 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072300 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-web" Jul 27 16:59:22.072338 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072307 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-metric" Jul 27 16:59:22.072338 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072312 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-metric" Jul 27 16:59:22.072338 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072322 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="alertmanager" Jul 27 16:59:22.072338 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072327 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="alertmanager" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072341 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072347 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072352 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0049617b-6725-46cf-a2d0-a02a59eb0751" containerName="console" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072357 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0049617b-6725-46cf-a2d0-a02a59eb0751" containerName="console" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072365 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="init-config-reloader" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072370 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="init-config-reloader" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072391 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="prom-label-proxy" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072396 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="prom-label-proxy" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072406 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="config-reloader" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072413 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="config-reloader" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072476 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="config-reloader" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072485 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-web" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072492 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="alertmanager" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072497 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0049617b-6725-46cf-a2d0-a02a59eb0751" containerName="console" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072503 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072509 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="kube-rbac-proxy-metric" Jul 27 16:59:22.072510 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.072514 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" containerName="prom-label-proxy" Jul 27 16:59:22.105134 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.105101 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:59:22.105264 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.105220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.108561 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108542 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 27 16:59:22.108700 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108670 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 27 16:59:22.108763 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108703 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rtz5f\"" Jul 27 16:59:22.108763 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108722 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 27 16:59:22.108851 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108805 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 27 16:59:22.108892 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108809 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 27 16:59:22.108993 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.108975 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 27 16:59:22.109108 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.109089 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 27 16:59:22.109177 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.109076 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 27 16:59:22.113801 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.113784 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 27 16:59:22.185689 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185644 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.185851 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185734 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4703f69c-24cc-47fa-bc3e-32000223c839-config-out\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.185851 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185772 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4703f69c-24cc-47fa-bc3e-32000223c839-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.185851 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185841 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-web-config\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.185960 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.185960 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186046 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.185984 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-config-volume\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186084 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.186044 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4703f69c-24cc-47fa-bc3e-32000223c839-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186084 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.186076 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186152 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.186100 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186152 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.186115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4703f69c-24cc-47fa-bc3e-32000223c839-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186152 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.186129 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42cww\" (UniqueName: \"kubernetes.io/projected/4703f69c-24cc-47fa-bc3e-32000223c839-kube-api-access-42cww\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.186152 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.186149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4703f69c-24cc-47fa-bc3e-32000223c839-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.197631 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.197603 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d06f7932-49bb-4e76-a6b8-ce4958a25b86" path="/var/lib/kubelet/pods/d06f7932-49bb-4e76-a6b8-ce4958a25b86/volumes" Jul 27 16:59:22.287683 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287604 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-web-config\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287683 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287645 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287742 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287762 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-config-volume\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4703f69c-24cc-47fa-bc3e-32000223c839-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287822 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.287852 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287837 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4703f69c-24cc-47fa-bc3e-32000223c839-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.288405 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.287859 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cww\" (UniqueName: \"kubernetes.io/projected/4703f69c-24cc-47fa-bc3e-32000223c839-kube-api-access-42cww\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.288481 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.288447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4703f69c-24cc-47fa-bc3e-32000223c839-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.288553 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.288532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.288601 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.288550 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4703f69c-24cc-47fa-bc3e-32000223c839-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.288601 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.288574 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4703f69c-24cc-47fa-bc3e-32000223c839-config-out\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.288703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.288603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4703f69c-24cc-47fa-bc3e-32000223c839-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.289341 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.289270 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4703f69c-24cc-47fa-bc3e-32000223c839-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.289533 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.289504 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4703f69c-24cc-47fa-bc3e-32000223c839-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291299 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291250 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-web-config\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291403 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291478 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-config-volume\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291478 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291437 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291821 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291801 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291912 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291895 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4703f69c-24cc-47fa-bc3e-32000223c839-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.291965 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291921 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.292007 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.291984 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4703f69c-24cc-47fa-bc3e-32000223c839-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.292552 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.292534 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4703f69c-24cc-47fa-bc3e-32000223c839-config-out\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.297056 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.297035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42cww\" (UniqueName: \"kubernetes.io/projected/4703f69c-24cc-47fa-bc3e-32000223c839-kube-api-access-42cww\") pod \"alertmanager-main-0\" (UID: \"4703f69c-24cc-47fa-bc3e-32000223c839\") " pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.415086 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.415049 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 27 16:59:22.577634 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:22.577606 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 27 16:59:22.578848 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:59:22.578827 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4703f69c_24cc_47fa_bc3e_32000223c839.slice/crio-3d1fd5eab78ada2fdb277d0a44b7de32454b8571363567a41486cfcf1e56d667 WatchSource:0}: Error finding container 3d1fd5eab78ada2fdb277d0a44b7de32454b8571363567a41486cfcf1e56d667: Status 404 returned error can't find the container with id 3d1fd5eab78ada2fdb277d0a44b7de32454b8571363567a41486cfcf1e56d667 Jul 27 16:59:23.018810 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:23.018775 2582 generic.go:356] "Generic (PLEG): container finished" podID="4703f69c-24cc-47fa-bc3e-32000223c839" containerID="beb8c533b4bdaed395edfe9bc7e24c9fe40f651b69f99a6060cf444510ced95a" exitCode=0 Jul 27 16:59:23.018810 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:23.018815 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerDied","Data":"beb8c533b4bdaed395edfe9bc7e24c9fe40f651b69f99a6060cf444510ced95a"} Jul 27 16:59:23.019235 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:23.018875 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"3d1fd5eab78ada2fdb277d0a44b7de32454b8571363567a41486cfcf1e56d667"} Jul 27 16:59:24.000492 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.000459 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg"] Jul 27 16:59:24.002972 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.002956 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.005946 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.005919 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 27 16:59:24.005946 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.005919 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kx527\"" Jul 27 16:59:24.006110 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.005924 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 27 16:59:24.006206 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.006192 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 27 16:59:24.006293 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.006277 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 27 16:59:24.006356 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.006343 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 27 16:59:24.012499 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.012478 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 27 16:59:24.016400 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.016365 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg"] Jul 27 16:59:24.027544 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.027512 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"39160d58f682ec35f6da9cde0f3301378c31f641dde756352e64752ee1cf5909"} Jul 27 16:59:24.027891 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.027568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"4caf1615111713b00ce36afd9a7fbd6b2d5a38ea5b4893feb39b10a78e6ac030"} Jul 27 16:59:24.027891 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.027593 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"a529833ee0186bc47db1a45f4859dd66dfeffb2cf593a1678fb9d9c212cf1bc5"} Jul 27 16:59:24.027891 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.027617 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"02edb14e8c65c1e4593952cc0b27d84027720c8bd77fa139fff18d516b0d8c2d"} Jul 27 16:59:24.027891 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.027643 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"99e1525519da6da80bf0ece1ace21a06ce177858f776f4af6c43b4ed9886b0db"} Jul 27 16:59:24.027891 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.027669 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4703f69c-24cc-47fa-bc3e-32000223c839","Type":"ContainerStarted","Data":"0a6948eeb01535fe662c3fc9ef46452e66eef3b05fa45e7e98cd29f506b63278"} Jul 27 16:59:24.055806 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.055750 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.042609085 podStartE2EDuration="2.055731888s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.122803ms" isStatefulPod=false podCreationTimestamp="2026-07-27 16:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:59:24.05460652 +0000 UTC m=+170.338765802" watchObservedRunningTime="2026-07-27 16:59:24.055731888 +0000 UTC m=+170.339891171" Jul 27 16:59:24.106179 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5ql6\" (UniqueName: \"kubernetes.io/projected/c2526be3-7e0c-4185-af23-9ce719dc42e5-kube-api-access-x5ql6\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106310 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106193 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-serving-certs-ca-bundle\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106310 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-metrics-client-ca\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106310 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106263 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106310 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106527 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106342 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-telemeter-client-tls\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106527 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-federate-client-tls\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.106527 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.106458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-secret-telemeter-client\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.213667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.213627 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-serving-certs-ca-bundle\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.213667 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.213671 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-metrics-client-ca\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.213920 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.213805 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.214092 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.214065 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.214522 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.214497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-telemeter-client-tls\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.214646 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.214584 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-federate-client-tls\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.214646 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.214625 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-secret-telemeter-client\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.214768 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.214710 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5ql6\" (UniqueName: \"kubernetes.io/projected/c2526be3-7e0c-4185-af23-9ce719dc42e5-kube-api-access-x5ql6\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.214819 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.214793 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-metrics-client-ca\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.215164 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.215140 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-serving-certs-ca-bundle\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.215371 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.215202 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2526be3-7e0c-4185-af23-9ce719dc42e5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.217232 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.217207 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.217992 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.217969 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-telemeter-client-tls\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.218088 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.218000 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-secret-telemeter-client\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.218291 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.218272 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c2526be3-7e0c-4185-af23-9ce719dc42e5-federate-client-tls\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.224675 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.224536 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5ql6\" (UniqueName: \"kubernetes.io/projected/c2526be3-7e0c-4185-af23-9ce719dc42e5-kube-api-access-x5ql6\") pod \"telemeter-client-6f9b5986d6-5g7jg\" (UID: \"c2526be3-7e0c-4185-af23-9ce719dc42e5\") " pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.313512 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.313430 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" Jul 27 16:59:24.448065 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:24.448040 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg"] Jul 27 16:59:24.449993 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:59:24.449963 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2526be3_7e0c_4185_af23_9ce719dc42e5.slice/crio-0d85f7da7ec58a2eff95cf96d7b8bb4f91705e923c3392c9001ccfdde735e7b4 WatchSource:0}: Error finding container 0d85f7da7ec58a2eff95cf96d7b8bb4f91705e923c3392c9001ccfdde735e7b4: Status 404 returned error can't find the container with id 0d85f7da7ec58a2eff95cf96d7b8bb4f91705e923c3392c9001ccfdde735e7b4 Jul 27 16:59:25.033229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:25.033193 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" event={"ID":"c2526be3-7e0c-4185-af23-9ce719dc42e5","Type":"ContainerStarted","Data":"0d85f7da7ec58a2eff95cf96d7b8bb4f91705e923c3392c9001ccfdde735e7b4"} Jul 27 16:59:27.045703 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.045662 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" event={"ID":"c2526be3-7e0c-4185-af23-9ce719dc42e5","Type":"ContainerStarted","Data":"e8208386c9c365d15cb4bb98c3f60b30dc9763bdd057b000d36795bd61732562"} Jul 27 16:59:27.046104 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.045731 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" event={"ID":"c2526be3-7e0c-4185-af23-9ce719dc42e5","Type":"ContainerStarted","Data":"2f301437b0c4bfb79b26f4b4f40f687c8e4819e521f1532930e8522b748f86fe"} Jul 27 16:59:27.046104 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.045760 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" event={"ID":"c2526be3-7e0c-4185-af23-9ce719dc42e5","Type":"ContainerStarted","Data":"f247b5435a5b8d772ab51d3d7d52b13ad79856fdb6f8ecfaaee66563f384e5e9"} Jul 27 16:59:27.073078 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.073026 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f9b5986d6-5g7jg" podStartSLOduration=2.153162347 podStartE2EDuration="4.073011559s" totalImagesPullingTime="1.919849212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:59:27.072550974 +0000 UTC m=+173.356710270" watchObservedRunningTime="2026-07-27 16:59:27.073011559 +0000 UTC m=+173.357170843" Jul 27 16:59:27.928482 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.928446 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cd49696b9-2d9wr"] Jul 27 16:59:27.930653 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.930634 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:27.947048 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:27.947018 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cd49696b9-2d9wr"] Jul 27 16:59:28.047770 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.047740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-service-ca\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.047770 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.047773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-trusted-ca-bundle\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.048229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.047801 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-oauth-serving-cert\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.048229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.047870 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-serving-cert\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.048229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.047973 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhcfk\" (UniqueName: \"kubernetes.io/projected/ecc3ae23-c95e-417f-8695-bfdffef205ee-kube-api-access-jhcfk\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.048229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.048035 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-config\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.048229 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.048058 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-oauth-config\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.148637 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.148589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-config\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.148637 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.148636 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-oauth-config\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.148890 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.148695 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-service-ca\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.148890 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.148724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-trusted-ca-bundle\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.148890 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.148800 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-oauth-serving-cert\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.149915 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.149880 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-config\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.150036 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.148839 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-serving-cert\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.150102 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.150040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-oauth-serving-cert\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.150212 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.150189 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-service-ca\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.150325 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.150045 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhcfk\" (UniqueName: \"kubernetes.io/projected/ecc3ae23-c95e-417f-8695-bfdffef205ee-kube-api-access-jhcfk\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.151496 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.151470 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-trusted-ca-bundle\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.152197 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.152176 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-serving-cert\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.157799 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.157774 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-oauth-config\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.158983 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.158945 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhcfk\" (UniqueName: \"kubernetes.io/projected/ecc3ae23-c95e-417f-8695-bfdffef205ee-kube-api-access-jhcfk\") pod \"console-5cd49696b9-2d9wr\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.240480 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.240393 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:28.371599 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:28.371566 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cd49696b9-2d9wr"] Jul 27 16:59:28.374985 ip-10-0-135-54 kubenswrapper[2582]: W0727 16:59:28.374956 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc3ae23_c95e_417f_8695_bfdffef205ee.slice/crio-f26e4f60ad8b9eec7aa0473ce11bc263ee3fd84231aa3cafc933082e1cec8feb WatchSource:0}: Error finding container f26e4f60ad8b9eec7aa0473ce11bc263ee3fd84231aa3cafc933082e1cec8feb: Status 404 returned error can't find the container with id f26e4f60ad8b9eec7aa0473ce11bc263ee3fd84231aa3cafc933082e1cec8feb Jul 27 16:59:29.056242 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:29.056205 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd49696b9-2d9wr" event={"ID":"ecc3ae23-c95e-417f-8695-bfdffef205ee","Type":"ContainerStarted","Data":"1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf"} Jul 27 16:59:29.056617 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:29.056255 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd49696b9-2d9wr" event={"ID":"ecc3ae23-c95e-417f-8695-bfdffef205ee","Type":"ContainerStarted","Data":"f26e4f60ad8b9eec7aa0473ce11bc263ee3fd84231aa3cafc933082e1cec8feb"} Jul 27 16:59:29.077008 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:29.076960 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cd49696b9-2d9wr" podStartSLOduration=2.076944524 podStartE2EDuration="2.076944524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 16:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 16:59:29.07642051 +0000 UTC m=+175.360579793" watchObservedRunningTime="2026-07-27 16:59:29.076944524 +0000 UTC m=+175.361103883" Jul 27 16:59:38.240828 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:38.240787 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:38.241271 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:38.240853 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:38.245938 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:38.245915 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:39.104165 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:39.104138 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 16:59:39.159550 ip-10-0-135-54 kubenswrapper[2582]: I0727 16:59:39.159510 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d7d7694b5-r2njg"] Jul 27 17:00:04.184946 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.184886 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d7d7694b5-r2njg" podUID="d157d9bc-8e5b-4052-bbcd-ecef49d4a276" containerName="console" containerID="cri-o://07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57" gracePeriod=15 Jul 27 17:00:04.437172 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.437102 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 17:00:04.566579 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566532 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf5h8\" (UniqueName: \"kubernetes.io/projected/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-kube-api-access-jf5h8\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.566793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566588 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-trusted-ca-bundle\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.566793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566630 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-oauth-serving-cert\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.566793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566647 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-serving-cert\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.566793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566690 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-oauth-config\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.566793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566727 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-config\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.566793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.566771 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-service-ca\") pod \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\" (UID: \"d157d9bc-8e5b-4052-bbcd-ecef49d4a276\") " Jul 27 17:00:04.567113 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.567068 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:00:04.567113 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.567088 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:00:04.567348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.567318 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-config" (OuterVolumeSpecName: "console-config") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:00:04.567466 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.567333 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-service-ca" (OuterVolumeSpecName: "service-ca") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:00:04.568974 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.568945 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:00:04.569079 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.568968 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:00:04.569079 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.568999 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-kube-api-access-jf5h8" (OuterVolumeSpecName: "kube-api-access-jf5h8") pod "d157d9bc-8e5b-4052-bbcd-ecef49d4a276" (UID: "d157d9bc-8e5b-4052-bbcd-ecef49d4a276"). InnerVolumeSpecName "kube-api-access-jf5h8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:00:04.668348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668293 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf5h8\" (UniqueName: \"kubernetes.io/projected/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-kube-api-access-jf5h8\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:04.668348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668340 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-trusted-ca-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:04.668348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668350 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-oauth-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:04.668348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668359 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:04.668348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668367 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-oauth-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:04.668348 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668389 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-console-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:04.668686 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:04.668398 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d157d9bc-8e5b-4052-bbcd-ecef49d4a276-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:00:05.210759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.210716 2582 generic.go:356] "Generic (PLEG): container finished" podID="d157d9bc-8e5b-4052-bbcd-ecef49d4a276" containerID="07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57" exitCode=0 Jul 27 17:00:05.211201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.210782 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d7d7694b5-r2njg" Jul 27 17:00:05.211201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.210797 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7d7694b5-r2njg" event={"ID":"d157d9bc-8e5b-4052-bbcd-ecef49d4a276","Type":"ContainerDied","Data":"07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57"} Jul 27 17:00:05.211201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.210850 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d7d7694b5-r2njg" event={"ID":"d157d9bc-8e5b-4052-bbcd-ecef49d4a276","Type":"ContainerDied","Data":"0ccddea0b9a7799db05f0c493cecb725da42167a2cdfb992da6aa775cde4eccc"} Jul 27 17:00:05.211201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.210872 2582 scope.go:117] "RemoveContainer" containerID="07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57" Jul 27 17:00:05.219643 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.219625 2582 scope.go:117] "RemoveContainer" containerID="07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57" Jul 27 17:00:05.219891 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:00:05.219873 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57\": container with ID starting with 07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57 not found: ID does not exist" containerID="07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57" Jul 27 17:00:05.219949 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.219899 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57"} err="failed to get container status \"07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57\": rpc error: code = NotFound desc = could not find container \"07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57\": container with ID starting with 07b5c533a3cc03a075f3d7af9abfc11f007e1f253a51fe25d9bd974e310e6f57 not found: ID does not exist" Jul 27 17:00:05.232993 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.232969 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d7d7694b5-r2njg"] Jul 27 17:00:05.237037 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:05.237016 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d7d7694b5-r2njg"] Jul 27 17:00:06.197343 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:00:06.197308 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d157d9bc-8e5b-4052-bbcd-ecef49d4a276" path="/var/lib/kubelet/pods/d157d9bc-8e5b-4052-bbcd-ecef49d4a276/volumes" Jul 27 17:01:14.831626 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.831590 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f55d6d855-t9wqs"] Jul 27 17:01:14.832059 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.831916 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d157d9bc-8e5b-4052-bbcd-ecef49d4a276" containerName="console" Jul 27 17:01:14.832059 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.831927 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d157d9bc-8e5b-4052-bbcd-ecef49d4a276" containerName="console" Jul 27 17:01:14.832059 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.831986 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d157d9bc-8e5b-4052-bbcd-ecef49d4a276" containerName="console" Jul 27 17:01:14.834906 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.834882 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.854402 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.854350 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f55d6d855-t9wqs"] Jul 27 17:01:14.880426 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880390 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghjc7\" (UniqueName: \"kubernetes.io/projected/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-kube-api-access-ghjc7\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.880573 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-oauth-config\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.880573 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880486 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-config\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.880573 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-trusted-ca-bundle\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.880676 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880576 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-oauth-serving-cert\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.880676 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-service-ca\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.880676 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.880644 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-serving-cert\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982011 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.981976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-serving-cert\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982176 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982017 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghjc7\" (UniqueName: \"kubernetes.io/projected/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-kube-api-access-ghjc7\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982176 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-oauth-config\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982276 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982182 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-config\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982276 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982231 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-trusted-ca-bundle\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982374 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-oauth-serving-cert\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982374 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-service-ca\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.982938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.982917 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-config\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.983032 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.983009 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-oauth-serving-cert\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.983071 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.983055 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-service-ca\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.983298 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.983281 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-trusted-ca-bundle\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.984948 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.984925 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-serving-cert\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.985076 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.985055 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-oauth-config\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:14.990877 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:14.990854 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghjc7\" (UniqueName: \"kubernetes.io/projected/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-kube-api-access-ghjc7\") pod \"console-6f55d6d855-t9wqs\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:15.144179 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:15.144135 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:15.281154 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:15.281109 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f55d6d855-t9wqs"] Jul 27 17:01:15.283461 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:01:15.283436 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2b7c21_1b8f_41cb_89dd_b730512f56e4.slice/crio-b522c8ea66cf2fe5b046eb31f01b7c4b03a85fce9c3ae67d319870590a4b62b2 WatchSource:0}: Error finding container b522c8ea66cf2fe5b046eb31f01b7c4b03a85fce9c3ae67d319870590a4b62b2: Status 404 returned error can't find the container with id b522c8ea66cf2fe5b046eb31f01b7c4b03a85fce9c3ae67d319870590a4b62b2 Jul 27 17:01:15.502910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:15.502820 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f55d6d855-t9wqs" event={"ID":"1e2b7c21-1b8f-41cb-89dd-b730512f56e4","Type":"ContainerStarted","Data":"f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746"} Jul 27 17:01:15.502910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:15.502870 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f55d6d855-t9wqs" event={"ID":"1e2b7c21-1b8f-41cb-89dd-b730512f56e4","Type":"ContainerStarted","Data":"b522c8ea66cf2fe5b046eb31f01b7c4b03a85fce9c3ae67d319870590a4b62b2"} Jul 27 17:01:15.525177 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:15.525123 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f55d6d855-t9wqs" podStartSLOduration=1.5251065590000001 podStartE2EDuration="1.525106559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:01:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:01:15.524193668 +0000 UTC m=+281.808352952" watchObservedRunningTime="2026-07-27 17:01:15.525106559 +0000 UTC m=+281.809265841" Jul 27 17:01:25.144659 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:25.144614 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:25.145139 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:25.144686 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:25.149493 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:25.149469 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:25.547692 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:25.547612 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:01:25.600686 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:25.600650 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cd49696b9-2d9wr"] Jul 27 17:01:34.046223 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:34.046197 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 27 17:01:50.622443 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.622366 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cd49696b9-2d9wr" podUID="ecc3ae23-c95e-417f-8695-bfdffef205ee" containerName="console" containerID="cri-o://1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf" gracePeriod=15 Jul 27 17:01:50.869000 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.868977 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 17:01:50.907304 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907232 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-config\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907304 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907268 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-oauth-config\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907304 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907297 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-service-ca\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907588 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907340 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-serving-cert\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907588 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907362 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhcfk\" (UniqueName: \"kubernetes.io/projected/ecc3ae23-c95e-417f-8695-bfdffef205ee-kube-api-access-jhcfk\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907588 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907458 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-oauth-serving-cert\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907588 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907517 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-trusted-ca-bundle\") pod \"ecc3ae23-c95e-417f-8695-bfdffef205ee\" (UID: \"ecc3ae23-c95e-417f-8695-bfdffef205ee\") " Jul 27 17:01:50.907798 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907667 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-config" (OuterVolumeSpecName: "console-config") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:01:50.907798 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907708 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-service-ca" (OuterVolumeSpecName: "service-ca") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:01:50.907905 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907826 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:50.907994 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.907962 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:50.908118 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.908013 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:01:50.908118 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.908077 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:01:50.909571 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.909548 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:01:50.909663 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.909606 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc3ae23-c95e-417f-8695-bfdffef205ee-kube-api-access-jhcfk" (OuterVolumeSpecName: "kube-api-access-jhcfk") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "kube-api-access-jhcfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:01:50.909909 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:50.909890 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ecc3ae23-c95e-417f-8695-bfdffef205ee" (UID: "ecc3ae23-c95e-417f-8695-bfdffef205ee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:01:51.008492 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.008463 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-trusted-ca-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:51.008492 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.008488 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-oauth-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:51.008492 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.008498 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc3ae23-c95e-417f-8695-bfdffef205ee-console-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:51.008703 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.008507 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhcfk\" (UniqueName: \"kubernetes.io/projected/ecc3ae23-c95e-417f-8695-bfdffef205ee-kube-api-access-jhcfk\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:51.008703 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.008516 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc3ae23-c95e-417f-8695-bfdffef205ee-oauth-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:01:51.653838 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.653804 2582 generic.go:356] "Generic (PLEG): container finished" podID="ecc3ae23-c95e-417f-8695-bfdffef205ee" containerID="1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf" exitCode=0 Jul 27 17:01:51.654283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.653847 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cd49696b9-2d9wr" Jul 27 17:01:51.654283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.653850 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd49696b9-2d9wr" event={"ID":"ecc3ae23-c95e-417f-8695-bfdffef205ee","Type":"ContainerDied","Data":"1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf"} Jul 27 17:01:51.654283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.653900 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cd49696b9-2d9wr" event={"ID":"ecc3ae23-c95e-417f-8695-bfdffef205ee","Type":"ContainerDied","Data":"f26e4f60ad8b9eec7aa0473ce11bc263ee3fd84231aa3cafc933082e1cec8feb"} Jul 27 17:01:51.654283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.653927 2582 scope.go:117] "RemoveContainer" containerID="1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf" Jul 27 17:01:51.663086 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.663069 2582 scope.go:117] "RemoveContainer" containerID="1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf" Jul 27 17:01:51.663347 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:01:51.663330 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf\": container with ID starting with 1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf not found: ID does not exist" containerID="1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf" Jul 27 17:01:51.663401 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.663356 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf"} err="failed to get container status \"1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf\": rpc error: code = NotFound desc = could not find container \"1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf\": container with ID starting with 1f78d03c85c15e582a0aff97e3d566a2263fa32c1636050dc2d0fcc10c5b58cf not found: ID does not exist" Jul 27 17:01:51.675599 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.675576 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cd49696b9-2d9wr"] Jul 27 17:01:51.679222 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:51.679199 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cd49696b9-2d9wr"] Jul 27 17:01:52.197852 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:52.197818 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecc3ae23-c95e-417f-8695-bfdffef205ee" path="/var/lib/kubelet/pods/ecc3ae23-c95e-417f-8695-bfdffef205ee/volumes" Jul 27 17:01:54.413688 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.413653 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp"] Jul 27 17:01:54.414051 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.413994 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecc3ae23-c95e-417f-8695-bfdffef205ee" containerName="console" Jul 27 17:01:54.414051 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.414005 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecc3ae23-c95e-417f-8695-bfdffef205ee" containerName="console" Jul 27 17:01:54.414119 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.414067 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecc3ae23-c95e-417f-8695-bfdffef205ee" containerName="console" Jul 27 17:01:54.418586 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.418563 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.421671 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.421651 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 17:01:54.422904 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.422887 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kcgp6\"" Jul 27 17:01:54.422904 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.422899 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 17:01:54.427568 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.427549 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp"] Jul 27 17:01:54.436282 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.436256 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.436453 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.436324 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.436453 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.436408 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2wwh\" (UniqueName: \"kubernetes.io/projected/37b1cd81-464f-40f2-9740-8a18519a7b90-kube-api-access-x2wwh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.536804 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.536766 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.536974 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.536853 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.536974 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.536887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wwh\" (UniqueName: \"kubernetes.io/projected/37b1cd81-464f-40f2-9740-8a18519a7b90-kube-api-access-x2wwh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.537219 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.537197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.537260 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.537197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.545886 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.545860 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wwh\" (UniqueName: \"kubernetes.io/projected/37b1cd81-464f-40f2-9740-8a18519a7b90-kube-api-access-x2wwh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.728178 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.728093 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:01:54.861518 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.861493 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp"] Jul 27 17:01:54.863970 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:01:54.863945 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b1cd81_464f_40f2_9740_8a18519a7b90.slice/crio-1546be01a1f3961b1a95cab96f868edaac6b9bc2a3aaccd96ca19838e01e1667 WatchSource:0}: Error finding container 1546be01a1f3961b1a95cab96f868edaac6b9bc2a3aaccd96ca19838e01e1667: Status 404 returned error can't find the container with id 1546be01a1f3961b1a95cab96f868edaac6b9bc2a3aaccd96ca19838e01e1667 Jul 27 17:01:54.865922 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:54.865907 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 17:01:55.675051 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:01:55.675015 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" event={"ID":"37b1cd81-464f-40f2-9740-8a18519a7b90","Type":"ContainerStarted","Data":"1546be01a1f3961b1a95cab96f868edaac6b9bc2a3aaccd96ca19838e01e1667"} Jul 27 17:02:00.701891 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:00.701799 2582 generic.go:356] "Generic (PLEG): container finished" podID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerID="6cf482af7e3099cbc6d8bc7d5a8408054c6027d40a0995dd9d39f7ec39c92f21" exitCode=0 Jul 27 17:02:00.701891 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:00.701865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" event={"ID":"37b1cd81-464f-40f2-9740-8a18519a7b90","Type":"ContainerDied","Data":"6cf482af7e3099cbc6d8bc7d5a8408054c6027d40a0995dd9d39f7ec39c92f21"} Jul 27 17:02:03.718108 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:03.718076 2582 generic.go:356] "Generic (PLEG): container finished" podID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerID="58453dd9982c2c1a5a400695f0e341ddd23be6db92ba864051d112b67754eb51" exitCode=0 Jul 27 17:02:03.718501 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:03.718160 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" event={"ID":"37b1cd81-464f-40f2-9740-8a18519a7b90","Type":"ContainerDied","Data":"58453dd9982c2c1a5a400695f0e341ddd23be6db92ba864051d112b67754eb51"} Jul 27 17:02:10.754426 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:10.754314 2582 generic.go:356] "Generic (PLEG): container finished" podID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerID="1918bab2d0760b37f6dff361e98a1b36caf88e434452eef9dc0792efbf4b9478" exitCode=0 Jul 27 17:02:10.754791 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:10.754418 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" event={"ID":"37b1cd81-464f-40f2-9740-8a18519a7b90","Type":"ContainerDied","Data":"1918bab2d0760b37f6dff361e98a1b36caf88e434452eef9dc0792efbf4b9478"} Jul 27 17:02:11.927870 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:11.927837 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:02:11.999426 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:11.999367 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-bundle\") pod \"37b1cd81-464f-40f2-9740-8a18519a7b90\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " Jul 27 17:02:11.999567 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:11.999440 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2wwh\" (UniqueName: \"kubernetes.io/projected/37b1cd81-464f-40f2-9740-8a18519a7b90-kube-api-access-x2wwh\") pod \"37b1cd81-464f-40f2-9740-8a18519a7b90\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " Jul 27 17:02:11.999567 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:11.999465 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-util\") pod \"37b1cd81-464f-40f2-9740-8a18519a7b90\" (UID: \"37b1cd81-464f-40f2-9740-8a18519a7b90\") " Jul 27 17:02:12.000159 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.000125 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-bundle" (OuterVolumeSpecName: "bundle") pod "37b1cd81-464f-40f2-9740-8a18519a7b90" (UID: "37b1cd81-464f-40f2-9740-8a18519a7b90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:02:12.001725 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.001705 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b1cd81-464f-40f2-9740-8a18519a7b90-kube-api-access-x2wwh" (OuterVolumeSpecName: "kube-api-access-x2wwh") pod "37b1cd81-464f-40f2-9740-8a18519a7b90" (UID: "37b1cd81-464f-40f2-9740-8a18519a7b90"). InnerVolumeSpecName "kube-api-access-x2wwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:02:12.003655 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.003632 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-util" (OuterVolumeSpecName: "util") pod "37b1cd81-464f-40f2-9740-8a18519a7b90" (UID: "37b1cd81-464f-40f2-9740-8a18519a7b90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:02:12.100326 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.100296 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:12.100326 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.100325 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2wwh\" (UniqueName: \"kubernetes.io/projected/37b1cd81-464f-40f2-9740-8a18519a7b90-kube-api-access-x2wwh\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:12.100543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.100337 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37b1cd81-464f-40f2-9740-8a18519a7b90-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:12.765021 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.764993 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" Jul 27 17:02:12.765195 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.764992 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540k6lxp" event={"ID":"37b1cd81-464f-40f2-9740-8a18519a7b90","Type":"ContainerDied","Data":"1546be01a1f3961b1a95cab96f868edaac6b9bc2a3aaccd96ca19838e01e1667"} Jul 27 17:02:12.765195 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:12.765114 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1546be01a1f3961b1a95cab96f868edaac6b9bc2a3aaccd96ca19838e01e1667" Jul 27 17:02:17.653073 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653038 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf"] Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653423 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="util" Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653437 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="util" Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653446 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="pull" Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653451 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="pull" Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653460 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="extract" Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653465 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="extract" Jul 27 17:02:17.653543 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.653521 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b1cd81-464f-40f2-9740-8a18519a7b90" containerName="extract" Jul 27 17:02:17.656269 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.656254 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.659202 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.659180 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lmx6z\"" Jul 27 17:02:17.659457 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.659434 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 27 17:02:17.659555 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.659496 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 27 17:02:17.667434 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.667410 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf"] Jul 27 17:02:17.752488 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.752450 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be749d19-4921-4fde-92eb-c9a86513e1b3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fg6lf\" (UID: \"be749d19-4921-4fde-92eb-c9a86513e1b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.752685 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.752525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn6rf\" (UniqueName: \"kubernetes.io/projected/be749d19-4921-4fde-92eb-c9a86513e1b3-kube-api-access-pn6rf\") pod \"cert-manager-operator-controller-manager-55c444bf86-fg6lf\" (UID: \"be749d19-4921-4fde-92eb-c9a86513e1b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.853747 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.853707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn6rf\" (UniqueName: \"kubernetes.io/projected/be749d19-4921-4fde-92eb-c9a86513e1b3-kube-api-access-pn6rf\") pod \"cert-manager-operator-controller-manager-55c444bf86-fg6lf\" (UID: \"be749d19-4921-4fde-92eb-c9a86513e1b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.853940 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.853775 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be749d19-4921-4fde-92eb-c9a86513e1b3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fg6lf\" (UID: \"be749d19-4921-4fde-92eb-c9a86513e1b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.854207 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.854188 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be749d19-4921-4fde-92eb-c9a86513e1b3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-fg6lf\" (UID: \"be749d19-4921-4fde-92eb-c9a86513e1b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.864193 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.864166 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn6rf\" (UniqueName: \"kubernetes.io/projected/be749d19-4921-4fde-92eb-c9a86513e1b3-kube-api-access-pn6rf\") pod \"cert-manager-operator-controller-manager-55c444bf86-fg6lf\" (UID: \"be749d19-4921-4fde-92eb-c9a86513e1b3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:17.965796 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:17.965716 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" Jul 27 17:02:18.104416 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:18.104371 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf"] Jul 27 17:02:18.106524 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:02:18.106498 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe749d19_4921_4fde_92eb_c9a86513e1b3.slice/crio-0a4d39501a2de1d7124c7ff29ca164f1c8b528c79e4ac1a710274476bd03d1d2 WatchSource:0}: Error finding container 0a4d39501a2de1d7124c7ff29ca164f1c8b528c79e4ac1a710274476bd03d1d2: Status 404 returned error can't find the container with id 0a4d39501a2de1d7124c7ff29ca164f1c8b528c79e4ac1a710274476bd03d1d2 Jul 27 17:02:18.792124 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:18.792089 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" event={"ID":"be749d19-4921-4fde-92eb-c9a86513e1b3","Type":"ContainerStarted","Data":"0a4d39501a2de1d7124c7ff29ca164f1c8b528c79e4ac1a710274476bd03d1d2"} Jul 27 17:02:19.798748 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:19.798713 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" event={"ID":"be749d19-4921-4fde-92eb-c9a86513e1b3","Type":"ContainerStarted","Data":"d7eecb40531c867c1c631874203b4ff2a748b888586e77d54f4f9e5c7542a48c"} Jul 27 17:02:19.827616 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:19.827238 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-fg6lf" podStartSLOduration=1.249176171 podStartE2EDuration="2.827216685s" totalImagesPullingTime="1.578040514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:02:19.820688974 +0000 UTC m=+346.104848256" watchObservedRunningTime="2026-07-27 17:02:19.827216685 +0000 UTC m=+346.111375969" Jul 27 17:02:21.442088 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.442045 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn"] Jul 27 17:02:21.444751 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.444728 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.448942 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.448922 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kcgp6\"" Jul 27 17:02:21.449127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.449116 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 17:02:21.450259 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.450243 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 17:02:21.468122 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.468095 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn"] Jul 27 17:02:21.586842 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.586802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.586842 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.586843 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2zrq\" (UniqueName: \"kubernetes.io/projected/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-kube-api-access-k2zrq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.587073 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.586930 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.688046 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.687993 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.688215 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.688086 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.688215 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.688104 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2zrq\" (UniqueName: \"kubernetes.io/projected/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-kube-api-access-k2zrq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.688429 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.688408 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.688471 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.688443 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.698823 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.698765 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2zrq\" (UniqueName: \"kubernetes.io/projected/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-kube-api-access-k2zrq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.753780 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.753739 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:21.897727 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:21.897693 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn"] Jul 27 17:02:21.900937 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:02:21.900898 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f727be9_9212_4ebc_9a92_31b6b61ae6d5.slice/crio-8d3d7e2cf752ec4ae0b852f14c60335e2062e981e52938e505885705422aa6a7 WatchSource:0}: Error finding container 8d3d7e2cf752ec4ae0b852f14c60335e2062e981e52938e505885705422aa6a7: Status 404 returned error can't find the container with id 8d3d7e2cf752ec4ae0b852f14c60335e2062e981e52938e505885705422aa6a7 Jul 27 17:02:22.817930 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:22.817891 2582 generic.go:356] "Generic (PLEG): container finished" podID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerID="d49d2eaa036d9cb6e32630569ab9626cb20d5a12ae87d9f2ee02b01b64ab7978" exitCode=0 Jul 27 17:02:22.818302 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:22.817992 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" event={"ID":"2f727be9-9212-4ebc-9a92-31b6b61ae6d5","Type":"ContainerDied","Data":"d49d2eaa036d9cb6e32630569ab9626cb20d5a12ae87d9f2ee02b01b64ab7978"} Jul 27 17:02:22.818302 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:22.818052 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" event={"ID":"2f727be9-9212-4ebc-9a92-31b6b61ae6d5","Type":"ContainerStarted","Data":"8d3d7e2cf752ec4ae0b852f14c60335e2062e981e52938e505885705422aa6a7"} Jul 27 17:02:28.852010 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:28.851975 2582 generic.go:356] "Generic (PLEG): container finished" podID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerID="4f2a3b9fe4733a312e3b83425d13b093a782884d1c6fb3beedb0df3c75d35612" exitCode=0 Jul 27 17:02:28.852491 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:28.852035 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" event={"ID":"2f727be9-9212-4ebc-9a92-31b6b61ae6d5","Type":"ContainerDied","Data":"4f2a3b9fe4733a312e3b83425d13b093a782884d1c6fb3beedb0df3c75d35612"} Jul 27 17:02:29.858441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:29.858406 2582 generic.go:356] "Generic (PLEG): container finished" podID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerID="2c5d2e08a96fec9a41142586b547320e6070878a4fbec527f4efd5076d567dcb" exitCode=0 Jul 27 17:02:29.858819 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:29.858467 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" event={"ID":"2f727be9-9212-4ebc-9a92-31b6b61ae6d5","Type":"ContainerDied","Data":"2c5d2e08a96fec9a41142586b547320e6070878a4fbec527f4efd5076d567dcb"} Jul 27 17:02:30.988228 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:30.988196 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:31.168529 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.168434 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-bundle\") pod \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " Jul 27 17:02:31.168529 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.168498 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2zrq\" (UniqueName: \"kubernetes.io/projected/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-kube-api-access-k2zrq\") pod \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " Jul 27 17:02:31.168761 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.168571 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-util\") pod \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\" (UID: \"2f727be9-9212-4ebc-9a92-31b6b61ae6d5\") " Jul 27 17:02:31.168970 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.168932 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-bundle" (OuterVolumeSpecName: "bundle") pod "2f727be9-9212-4ebc-9a92-31b6b61ae6d5" (UID: "2f727be9-9212-4ebc-9a92-31b6b61ae6d5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:02:31.170746 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.170716 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-kube-api-access-k2zrq" (OuterVolumeSpecName: "kube-api-access-k2zrq") pod "2f727be9-9212-4ebc-9a92-31b6b61ae6d5" (UID: "2f727be9-9212-4ebc-9a92-31b6b61ae6d5"). InnerVolumeSpecName "kube-api-access-k2zrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:02:31.172954 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.172927 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-util" (OuterVolumeSpecName: "util") pod "2f727be9-9212-4ebc-9a92-31b6b61ae6d5" (UID: "2f727be9-9212-4ebc-9a92-31b6b61ae6d5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:02:31.269978 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.269942 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:31.269978 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.269973 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2zrq\" (UniqueName: \"kubernetes.io/projected/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-kube-api-access-k2zrq\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:31.269978 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.269986 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f727be9-9212-4ebc-9a92-31b6b61ae6d5-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:31.869692 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.869657 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" Jul 27 17:02:31.869872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.869662 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhcqbn" event={"ID":"2f727be9-9212-4ebc-9a92-31b6b61ae6d5","Type":"ContainerDied","Data":"8d3d7e2cf752ec4ae0b852f14c60335e2062e981e52938e505885705422aa6a7"} Jul 27 17:02:31.869872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:31.869776 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d3d7e2cf752ec4ae0b852f14c60335e2062e981e52938e505885705422aa6a7" Jul 27 17:02:38.767283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767248 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9"] Jul 27 17:02:38.767661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767579 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="pull" Jul 27 17:02:38.767661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767590 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="pull" Jul 27 17:02:38.767661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767607 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="extract" Jul 27 17:02:38.767661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767612 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="extract" Jul 27 17:02:38.767661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767620 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="util" Jul 27 17:02:38.767661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767626 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="util" Jul 27 17:02:38.767846 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.767679 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f727be9-9212-4ebc-9a92-31b6b61ae6d5" containerName="extract" Jul 27 17:02:38.771913 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.771897 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:38.775079 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.775060 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 27 17:02:38.775179 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.775137 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wtj85\"" Jul 27 17:02:38.775255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.775241 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 27 17:02:38.780448 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.780429 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9"] Jul 27 17:02:38.936299 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.936262 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwqln\" (UniqueName: \"kubernetes.io/projected/4f947712-1b29-4ee2-a2b4-3b05c2cac568-kube-api-access-pwqln\") pod \"openshift-lws-operator-fd8ccff4c-hpss9\" (UID: \"4f947712-1b29-4ee2-a2b4-3b05c2cac568\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:38.936524 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:38.936427 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f947712-1b29-4ee2-a2b4-3b05c2cac568-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hpss9\" (UID: \"4f947712-1b29-4ee2-a2b4-3b05c2cac568\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:39.037020 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.036931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f947712-1b29-4ee2-a2b4-3b05c2cac568-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hpss9\" (UID: \"4f947712-1b29-4ee2-a2b4-3b05c2cac568\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:39.037172 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.037022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwqln\" (UniqueName: \"kubernetes.io/projected/4f947712-1b29-4ee2-a2b4-3b05c2cac568-kube-api-access-pwqln\") pod \"openshift-lws-operator-fd8ccff4c-hpss9\" (UID: \"4f947712-1b29-4ee2-a2b4-3b05c2cac568\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:39.037327 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.037305 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f947712-1b29-4ee2-a2b4-3b05c2cac568-tmp\") pod \"openshift-lws-operator-fd8ccff4c-hpss9\" (UID: \"4f947712-1b29-4ee2-a2b4-3b05c2cac568\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:39.047586 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.047558 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwqln\" (UniqueName: \"kubernetes.io/projected/4f947712-1b29-4ee2-a2b4-3b05c2cac568-kube-api-access-pwqln\") pod \"openshift-lws-operator-fd8ccff4c-hpss9\" (UID: \"4f947712-1b29-4ee2-a2b4-3b05c2cac568\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:39.089344 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.089311 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" Jul 27 17:02:39.231255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.231226 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9"] Jul 27 17:02:39.232860 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:02:39.232834 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f947712_1b29_4ee2_a2b4_3b05c2cac568.slice/crio-69fa2d575917327262c5f7c96712e1990684dfea77624d92d1370899e40e8702 WatchSource:0}: Error finding container 69fa2d575917327262c5f7c96712e1990684dfea77624d92d1370899e40e8702: Status 404 returned error can't find the container with id 69fa2d575917327262c5f7c96712e1990684dfea77624d92d1370899e40e8702 Jul 27 17:02:39.906337 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:39.906305 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" event={"ID":"4f947712-1b29-4ee2-a2b4-3b05c2cac568","Type":"ContainerStarted","Data":"69fa2d575917327262c5f7c96712e1990684dfea77624d92d1370899e40e8702"} Jul 27 17:02:42.922822 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:42.922787 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" event={"ID":"4f947712-1b29-4ee2-a2b4-3b05c2cac568","Type":"ContainerStarted","Data":"dfd21364e0ccf38928983d6db54cb636bfa76ca1d99e8e24ed6878ed8c29c997"} Jul 27 17:02:42.942354 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:42.942301 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-hpss9" podStartSLOduration=1.852187479 podStartE2EDuration="4.942286414s" totalImagesPullingTime="3.090098935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:02:42.941261431 +0000 UTC m=+369.225420713" watchObservedRunningTime="2026-07-27 17:02:42.942286414 +0000 UTC m=+369.226445696" Jul 27 17:02:45.822832 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.822791 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh"] Jul 27 17:02:45.825475 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.825453 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.828875 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.828855 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kcgp6\"" Jul 27 17:02:45.829327 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.829310 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 17:02:45.830252 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.830227 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 17:02:45.833465 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.833439 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh"] Jul 27 17:02:45.893254 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.893217 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.893254 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.893255 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8754p\" (UniqueName: \"kubernetes.io/projected/933dc12d-0777-4495-a692-ba47e790a861-kube-api-access-8754p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.893476 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.893325 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.994419 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.994364 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.994579 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.994424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8754p\" (UniqueName: \"kubernetes.io/projected/933dc12d-0777-4495-a692-ba47e790a861-kube-api-access-8754p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.994579 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.994487 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.994766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.994749 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:45.994808 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:45.994780 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:46.003613 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:46.003586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8754p\" (UniqueName: \"kubernetes.io/projected/933dc12d-0777-4495-a692-ba47e790a861-kube-api-access-8754p\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:46.136608 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:46.136572 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:46.275672 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:46.275643 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh"] Jul 27 17:02:46.278171 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:02:46.278144 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933dc12d_0777_4495_a692_ba47e790a861.slice/crio-4d7cc4d0b92e74bc27b7a9577df53cd936f8482ca7675d21974c867ac7b98ad5 WatchSource:0}: Error finding container 4d7cc4d0b92e74bc27b7a9577df53cd936f8482ca7675d21974c867ac7b98ad5: Status 404 returned error can't find the container with id 4d7cc4d0b92e74bc27b7a9577df53cd936f8482ca7675d21974c867ac7b98ad5 Jul 27 17:02:46.941491 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:46.941455 2582 generic.go:356] "Generic (PLEG): container finished" podID="933dc12d-0777-4495-a692-ba47e790a861" containerID="25977356eab8130036ceef6a6029fd17190e9ed0af89c5c7ad2f1ceee130c58a" exitCode=0 Jul 27 17:02:46.941896 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:46.941531 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" event={"ID":"933dc12d-0777-4495-a692-ba47e790a861","Type":"ContainerDied","Data":"25977356eab8130036ceef6a6029fd17190e9ed0af89c5c7ad2f1ceee130c58a"} Jul 27 17:02:46.941896 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:46.941567 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" event={"ID":"933dc12d-0777-4495-a692-ba47e790a861","Type":"ContainerStarted","Data":"4d7cc4d0b92e74bc27b7a9577df53cd936f8482ca7675d21974c867ac7b98ad5"} Jul 27 17:02:48.952282 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:48.952249 2582 generic.go:356] "Generic (PLEG): container finished" podID="933dc12d-0777-4495-a692-ba47e790a861" containerID="f6f6b9ff331f6545a084a7f1896bed0d93e7c6a0d45a72c1ef2c9737793622ba" exitCode=0 Jul 27 17:02:48.952658 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:48.952333 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" event={"ID":"933dc12d-0777-4495-a692-ba47e790a861","Type":"ContainerDied","Data":"f6f6b9ff331f6545a084a7f1896bed0d93e7c6a0d45a72c1ef2c9737793622ba"} Jul 27 17:02:49.959308 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:49.959273 2582 generic.go:356] "Generic (PLEG): container finished" podID="933dc12d-0777-4495-a692-ba47e790a861" containerID="8287e59f87836f08edf757d5e0eba8055ffbf35a05174c72248853ba6eba6d14" exitCode=0 Jul 27 17:02:49.959700 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:49.959332 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" event={"ID":"933dc12d-0777-4495-a692-ba47e790a861","Type":"ContainerDied","Data":"8287e59f87836f08edf757d5e0eba8055ffbf35a05174c72248853ba6eba6d14"} Jul 27 17:02:51.101806 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.101783 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:51.138568 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.138531 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-bundle\") pod \"933dc12d-0777-4495-a692-ba47e790a861\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " Jul 27 17:02:51.138568 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.138572 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-util\") pod \"933dc12d-0777-4495-a692-ba47e790a861\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " Jul 27 17:02:51.138790 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.138640 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8754p\" (UniqueName: \"kubernetes.io/projected/933dc12d-0777-4495-a692-ba47e790a861-kube-api-access-8754p\") pod \"933dc12d-0777-4495-a692-ba47e790a861\" (UID: \"933dc12d-0777-4495-a692-ba47e790a861\") " Jul 27 17:02:51.139297 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.139275 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-bundle" (OuterVolumeSpecName: "bundle") pod "933dc12d-0777-4495-a692-ba47e790a861" (UID: "933dc12d-0777-4495-a692-ba47e790a861"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:02:51.140997 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.140973 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933dc12d-0777-4495-a692-ba47e790a861-kube-api-access-8754p" (OuterVolumeSpecName: "kube-api-access-8754p") pod "933dc12d-0777-4495-a692-ba47e790a861" (UID: "933dc12d-0777-4495-a692-ba47e790a861"). InnerVolumeSpecName "kube-api-access-8754p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:02:51.144639 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.144614 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-util" (OuterVolumeSpecName: "util") pod "933dc12d-0777-4495-a692-ba47e790a861" (UID: "933dc12d-0777-4495-a692-ba47e790a861"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:02:51.239601 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.239523 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8754p\" (UniqueName: \"kubernetes.io/projected/933dc12d-0777-4495-a692-ba47e790a861-kube-api-access-8754p\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:51.239601 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.239552 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:51.239601 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.239562 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/933dc12d-0777-4495-a692-ba47e790a861-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:02:51.978928 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.978900 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" Jul 27 17:02:51.979115 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.978896 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftg2vh" event={"ID":"933dc12d-0777-4495-a692-ba47e790a861","Type":"ContainerDied","Data":"4d7cc4d0b92e74bc27b7a9577df53cd936f8482ca7675d21974c867ac7b98ad5"} Jul 27 17:02:51.979115 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:51.979024 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d7cc4d0b92e74bc27b7a9577df53cd936f8482ca7675d21974c867ac7b98ad5" Jul 27 17:02:58.672216 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672171 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq"] Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672603 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="extract" Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672617 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="extract" Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672627 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="pull" Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672633 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="pull" Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672648 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="util" Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672654 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="util" Jul 27 17:02:58.672724 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.672714 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="933dc12d-0777-4495-a692-ba47e790a861" containerName="extract" Jul 27 17:02:58.674852 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.674834 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.678116 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.678093 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 27 17:02:58.678233 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.678216 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h676l\"" Jul 27 17:02:58.678305 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.678240 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 27 17:02:58.678305 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.678291 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 27 17:02:58.678641 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.678623 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 27 17:02:58.705239 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.705204 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq"] Jul 27 17:02:58.805502 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.805465 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ba683be-eb69-4a31-88bb-682ae04294ae-manifests\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.805677 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.805511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ba683be-eb69-4a31-88bb-682ae04294ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.805677 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.805538 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba683be-eb69-4a31-88bb-682ae04294ae-tmp\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.805677 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.805610 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shkk5\" (UniqueName: \"kubernetes.io/projected/5ba683be-eb69-4a31-88bb-682ae04294ae-kube-api-access-shkk5\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.805788 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.805691 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ba683be-eb69-4a31-88bb-682ae04294ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.907104 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.907062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shkk5\" (UniqueName: \"kubernetes.io/projected/5ba683be-eb69-4a31-88bb-682ae04294ae-kube-api-access-shkk5\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.907302 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.907126 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ba683be-eb69-4a31-88bb-682ae04294ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.907302 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.907181 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ba683be-eb69-4a31-88bb-682ae04294ae-manifests\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.907302 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.907224 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ba683be-eb69-4a31-88bb-682ae04294ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.907302 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.907249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba683be-eb69-4a31-88bb-682ae04294ae-tmp\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.907658 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.907637 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ba683be-eb69-4a31-88bb-682ae04294ae-manifests\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.909581 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.909558 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ba683be-eb69-4a31-88bb-682ae04294ae-tmp\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.909748 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.909726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ba683be-eb69-4a31-88bb-682ae04294ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.909788 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.909769 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ba683be-eb69-4a31-88bb-682ae04294ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.916878 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.916853 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shkk5\" (UniqueName: \"kubernetes.io/projected/5ba683be-eb69-4a31-88bb-682ae04294ae-kube-api-access-shkk5\") pod \"opendatahub-operator-controller-manager-565c484996-4n2zq\" (UID: \"5ba683be-eb69-4a31-88bb-682ae04294ae\") " pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:58.985870 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:58.985785 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:02:59.133942 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:02:59.133900 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq"] Jul 27 17:02:59.160817 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:02:59.160789 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba683be_eb69_4a31_88bb_682ae04294ae.slice/crio-7b18d083bd3e4c25cdefe3f4c4adb303ffc20174a8a2e15f344228800cf1fa51 WatchSource:0}: Error finding container 7b18d083bd3e4c25cdefe3f4c4adb303ffc20174a8a2e15f344228800cf1fa51: Status 404 returned error can't find the container with id 7b18d083bd3e4c25cdefe3f4c4adb303ffc20174a8a2e15f344228800cf1fa51 Jul 27 17:03:00.018823 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:00.018784 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" event={"ID":"5ba683be-eb69-4a31-88bb-682ae04294ae","Type":"ContainerStarted","Data":"7b18d083bd3e4c25cdefe3f4c4adb303ffc20174a8a2e15f344228800cf1fa51"} Jul 27 17:03:03.036431 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:03.036373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" event={"ID":"5ba683be-eb69-4a31-88bb-682ae04294ae","Type":"ContainerStarted","Data":"a547bade311ea39fbecca30add6acf1de465f98475bbcef0de139cbf1fb0e7ea"} Jul 27 17:03:04.042956 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:04.042920 2582 generic.go:356] "Generic (PLEG): container finished" podID="5ba683be-eb69-4a31-88bb-682ae04294ae" containerID="a547bade311ea39fbecca30add6acf1de465f98475bbcef0de139cbf1fb0e7ea" exitCode=0 Jul 27 17:03:04.043418 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:04.043012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" event={"ID":"5ba683be-eb69-4a31-88bb-682ae04294ae","Type":"ContainerDied","Data":"a547bade311ea39fbecca30add6acf1de465f98475bbcef0de139cbf1fb0e7ea"} Jul 27 17:03:07.059220 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:07.059181 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" event={"ID":"5ba683be-eb69-4a31-88bb-682ae04294ae","Type":"ContainerStarted","Data":"f3557a7375bd69822e1910776a6c7a91f3b8999edb4b00f403b25e8357339246"} Jul 27 17:03:07.059683 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:07.059317 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:03:07.085679 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:07.085636 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" podStartSLOduration=2.84137143 podStartE2EDuration="9.085622175s" totalImagesPullingTime="5.210639936s" totalInitContainerRuntime="1.033610809s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:02:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:03:07.082472946 +0000 UTC m=+393.366632230" watchObservedRunningTime="2026-07-27 17:03:07.085622175 +0000 UTC m=+393.369781458" Jul 27 17:03:18.066508 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.066472 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-565c484996-4n2zq" Jul 27 17:03:18.272987 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.272945 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-b56999664-2r4kj"] Jul 27 17:03:18.283775 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.283748 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.285158 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.285132 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b56999664-2r4kj"] Jul 27 17:03:18.286671 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.286648 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 27 17:03:18.286800 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.286679 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 27 17:03:18.286968 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.286948 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 27 17:03:18.287131 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.287115 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gvlzk\"" Jul 27 17:03:18.379132 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.379094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/afbc6d9f-d48f-48bb-b211-9c91126981d0-metrics-cert\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.379300 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.379135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6bz7\" (UniqueName: \"kubernetes.io/projected/afbc6d9f-d48f-48bb-b211-9c91126981d0-kube-api-access-z6bz7\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.379300 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.379206 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/afbc6d9f-d48f-48bb-b211-9c91126981d0-manager-config\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.379300 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.379278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afbc6d9f-d48f-48bb-b211-9c91126981d0-cert\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.479788 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.479749 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/afbc6d9f-d48f-48bb-b211-9c91126981d0-metrics-cert\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.479788 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.479791 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6bz7\" (UniqueName: \"kubernetes.io/projected/afbc6d9f-d48f-48bb-b211-9c91126981d0-kube-api-access-z6bz7\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.479989 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.479827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/afbc6d9f-d48f-48bb-b211-9c91126981d0-manager-config\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.479989 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.479856 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afbc6d9f-d48f-48bb-b211-9c91126981d0-cert\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.480534 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.480509 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/afbc6d9f-d48f-48bb-b211-9c91126981d0-manager-config\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.482278 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.482253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/afbc6d9f-d48f-48bb-b211-9c91126981d0-metrics-cert\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.482367 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.482306 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/afbc6d9f-d48f-48bb-b211-9c91126981d0-cert\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.487953 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.487934 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6bz7\" (UniqueName: \"kubernetes.io/projected/afbc6d9f-d48f-48bb-b211-9c91126981d0-kube-api-access-z6bz7\") pod \"lws-controller-manager-b56999664-2r4kj\" (UID: \"afbc6d9f-d48f-48bb-b211-9c91126981d0\") " pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.594785 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.594743 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:18.728704 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:18.728676 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-b56999664-2r4kj"] Jul 27 17:03:18.731218 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:03:18.731194 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafbc6d9f_d48f_48bb_b211_9c91126981d0.slice/crio-1d53df509554c211a61beb8f144f52c735b3dd41834e16c277140124663f37b6 WatchSource:0}: Error finding container 1d53df509554c211a61beb8f144f52c735b3dd41834e16c277140124663f37b6: Status 404 returned error can't find the container with id 1d53df509554c211a61beb8f144f52c735b3dd41834e16c277140124663f37b6 Jul 27 17:03:19.122309 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:19.122276 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" event={"ID":"afbc6d9f-d48f-48bb-b211-9c91126981d0","Type":"ContainerStarted","Data":"1d53df509554c211a61beb8f144f52c735b3dd41834e16c277140124663f37b6"} Jul 27 17:03:23.142964 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:23.142928 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" event={"ID":"afbc6d9f-d48f-48bb-b211-9c91126981d0","Type":"ContainerStarted","Data":"58d6965cc8ce5276a6b1c4c4375b57a6ec53acc499c775a0702965b5c2e7ebd5"} Jul 27 17:03:23.143473 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:23.143118 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:23.166237 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:23.166186 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" podStartSLOduration=1.6056481150000002 podStartE2EDuration="5.166172554s" totalImagesPullingTime="3.560524439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:03:23.164651583 +0000 UTC m=+409.448810867" watchObservedRunningTime="2026-07-27 17:03:23.166172554 +0000 UTC m=+409.450331887" Jul 27 17:03:34.151444 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:34.151412 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-b56999664-2r4kj" Jul 27 17:03:39.422070 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.422027 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb"] Jul 27 17:03:39.432408 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.432373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.435572 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.435547 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 27 17:03:39.435709 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.435607 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-z8x95\"" Jul 27 17:03:39.436641 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.436617 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb"] Jul 27 17:03:39.466365 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466332 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466541 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466370 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jkvv\" (UniqueName: \"kubernetes.io/projected/f488af11-04f3-48ee-ad5c-dbad18742cb8-kube-api-access-6jkvv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466541 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466541 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466541 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466529 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466739 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466739 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466604 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f488af11-04f3-48ee-ad5c-dbad18742cb8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466739 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466673 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.466739 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.466726 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568142 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568101 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568142 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568145 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f488af11-04f3-48ee-ad5c-dbad18742cb8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568312 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568335 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jkvv\" (UniqueName: \"kubernetes.io/projected/f488af11-04f3-48ee-ad5c-dbad18742cb8-kube-api-access-6jkvv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568441 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568830 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568564 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568830 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568690 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568830 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568759 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.568952 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.568886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.569113 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.569095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f488af11-04f3-48ee-ad5c-dbad18742cb8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.570539 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.570516 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.570659 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.570641 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.576716 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.576692 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f488af11-04f3-48ee-ad5c-dbad18742cb8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.576870 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.576854 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jkvv\" (UniqueName: \"kubernetes.io/projected/f488af11-04f3-48ee-ad5c-dbad18742cb8-kube-api-access-6jkvv\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb\" (UID: \"f488af11-04f3-48ee-ad5c-dbad18742cb8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.744920 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.744810 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:03:39.886729 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:39.886695 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb"] Jul 27 17:03:39.888275 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:03:39.888235 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf488af11_04f3_48ee_ad5c_dbad18742cb8.slice/crio-9e0b8853d962c365c5e1cb16e7a977f32ce354cd0628c5694d5e547c4a92359d WatchSource:0}: Error finding container 9e0b8853d962c365c5e1cb16e7a977f32ce354cd0628c5694d5e547c4a92359d: Status 404 returned error can't find the container with id 9e0b8853d962c365c5e1cb16e7a977f32ce354cd0628c5694d5e547c4a92359d Jul 27 17:03:40.230501 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:03:40.230465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" event={"ID":"f488af11-04f3-48ee-ad5c-dbad18742cb8","Type":"ContainerStarted","Data":"9e0b8853d962c365c5e1cb16e7a977f32ce354cd0628c5694d5e547c4a92359d"} Jul 27 17:04:13.737765 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:13.737718 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:04:13.738184 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:13.737816 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:04:13.738184 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:13.737897 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:04:14.394620 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:14.394577 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" event={"ID":"f488af11-04f3-48ee-ad5c-dbad18742cb8","Type":"ContainerStarted","Data":"1ebe73d56b372de582dbde9c256cfc8c38359116361b33941604c2f12c0e4a1d"} Jul 27 17:04:14.418457 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:14.418403 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" podStartSLOduration=1.571486654 podStartE2EDuration="35.418387449s" totalImagesPullingTime="33.846900795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:04:14.416648447 +0000 UTC m=+460.700807731" watchObservedRunningTime="2026-07-27 17:04:14.418387449 +0000 UTC m=+460.702546723" Jul 27 17:04:14.745650 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:14.745559 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:04:14.751028 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:14.750997 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:04:15.399682 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:15.399648 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:04:15.400509 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:15.400492 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb" Jul 27 17:04:21.738006 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.737968 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7b2hr"] Jul 27 17:04:21.743371 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.743325 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:21.746618 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.746589 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 27 17:04:21.746909 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.746894 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 27 17:04:21.748373 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.748123 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-v4qd6\"" Jul 27 17:04:21.757806 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.753697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7b2hr"] Jul 27 17:04:21.848792 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.848754 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mt4m\" (UniqueName: \"kubernetes.io/projected/bb56e6ff-b2bd-4186-8020-68cc37d41447-kube-api-access-7mt4m\") pod \"kuadrant-operator-catalog-7b2hr\" (UID: \"bb56e6ff-b2bd-4186-8020-68cc37d41447\") " pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:21.949502 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.949464 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mt4m\" (UniqueName: \"kubernetes.io/projected/bb56e6ff-b2bd-4186-8020-68cc37d41447-kube-api-access-7mt4m\") pod \"kuadrant-operator-catalog-7b2hr\" (UID: \"bb56e6ff-b2bd-4186-8020-68cc37d41447\") " pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:21.958274 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:21.958242 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mt4m\" (UniqueName: \"kubernetes.io/projected/bb56e6ff-b2bd-4186-8020-68cc37d41447-kube-api-access-7mt4m\") pod \"kuadrant-operator-catalog-7b2hr\" (UID: \"bb56e6ff-b2bd-4186-8020-68cc37d41447\") " pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:22.056956 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.056870 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:22.104555 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.104514 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7b2hr"] Jul 27 17:04:22.190969 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.190944 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7b2hr"] Jul 27 17:04:22.192915 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:22.192875 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb56e6ff_b2bd_4186_8020_68cc37d41447.slice/crio-3df1d9eb6c18edcd8b4ab4d1a29def75db20f818d4e225aa4ed1be8c6f874dfe WatchSource:0}: Error finding container 3df1d9eb6c18edcd8b4ab4d1a29def75db20f818d4e225aa4ed1be8c6f874dfe: Status 404 returned error can't find the container with id 3df1d9eb6c18edcd8b4ab4d1a29def75db20f818d4e225aa4ed1be8c6f874dfe Jul 27 17:04:22.317434 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.317335 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh5mt"] Jul 27 17:04:22.321851 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.321834 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:22.340042 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.340013 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh5mt"] Jul 27 17:04:22.353987 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.353950 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqfl\" (UniqueName: \"kubernetes.io/projected/0abc3f2a-bb70-4126-a517-279465b4e124-kube-api-access-pqqfl\") pod \"kuadrant-operator-catalog-gh5mt\" (UID: \"0abc3f2a-bb70-4126-a517-279465b4e124\") " pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:22.435809 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.435774 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" event={"ID":"bb56e6ff-b2bd-4186-8020-68cc37d41447","Type":"ContainerStarted","Data":"3df1d9eb6c18edcd8b4ab4d1a29def75db20f818d4e225aa4ed1be8c6f874dfe"} Jul 27 17:04:22.455351 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.455318 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqfl\" (UniqueName: \"kubernetes.io/projected/0abc3f2a-bb70-4126-a517-279465b4e124-kube-api-access-pqqfl\") pod \"kuadrant-operator-catalog-gh5mt\" (UID: \"0abc3f2a-bb70-4126-a517-279465b4e124\") " pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:22.465037 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.465012 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqfl\" (UniqueName: \"kubernetes.io/projected/0abc3f2a-bb70-4126-a517-279465b4e124-kube-api-access-pqqfl\") pod \"kuadrant-operator-catalog-gh5mt\" (UID: \"0abc3f2a-bb70-4126-a517-279465b4e124\") " pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:22.631054 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.631015 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:22.763472 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:22.763448 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gh5mt"] Jul 27 17:04:22.765241 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:22.765214 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abc3f2a_bb70_4126_a517_279465b4e124.slice/crio-005c26414bb63890d1e148858665f2b8feed6002376a27d0efdf4dda598194ce WatchSource:0}: Error finding container 005c26414bb63890d1e148858665f2b8feed6002376a27d0efdf4dda598194ce: Status 404 returned error can't find the container with id 005c26414bb63890d1e148858665f2b8feed6002376a27d0efdf4dda598194ce Jul 27 17:04:23.442333 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:23.442301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" event={"ID":"0abc3f2a-bb70-4126-a517-279465b4e124","Type":"ContainerStarted","Data":"005c26414bb63890d1e148858665f2b8feed6002376a27d0efdf4dda598194ce"} Jul 27 17:04:25.455762 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.455665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" event={"ID":"bb56e6ff-b2bd-4186-8020-68cc37d41447","Type":"ContainerStarted","Data":"d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2"} Jul 27 17:04:25.455762 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.455727 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" podUID="bb56e6ff-b2bd-4186-8020-68cc37d41447" containerName="registry-server" containerID="cri-o://d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2" gracePeriod=2 Jul 27 17:04:25.457357 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.457331 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" event={"ID":"0abc3f2a-bb70-4126-a517-279465b4e124","Type":"ContainerStarted","Data":"c8f651698c7935b604bf794aa6683032711d8b8346d9fa602ca3d92951ed5905"} Jul 27 17:04:25.473869 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.473806 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" podStartSLOduration=1.461999654 podStartE2EDuration="4.473786396s" totalImagesPullingTime="3.011786742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:04:25.4725269 +0000 UTC m=+471.756686184" watchObservedRunningTime="2026-07-27 17:04:25.473786396 +0000 UTC m=+471.757945679" Jul 27 17:04:25.490586 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.490540 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" podStartSLOduration=1.055439737 podStartE2EDuration="3.49052583s" totalImagesPullingTime="2.435086093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:04:25.488366859 +0000 UTC m=+471.772526144" watchObservedRunningTime="2026-07-27 17:04:25.49052583 +0000 UTC m=+471.774685107" Jul 27 17:04:25.711425 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.711343 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:25.788370 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.788335 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mt4m\" (UniqueName: \"kubernetes.io/projected/bb56e6ff-b2bd-4186-8020-68cc37d41447-kube-api-access-7mt4m\") pod \"bb56e6ff-b2bd-4186-8020-68cc37d41447\" (UID: \"bb56e6ff-b2bd-4186-8020-68cc37d41447\") " Jul 27 17:04:25.790502 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.790473 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb56e6ff-b2bd-4186-8020-68cc37d41447-kube-api-access-7mt4m" (OuterVolumeSpecName: "kube-api-access-7mt4m") pod "bb56e6ff-b2bd-4186-8020-68cc37d41447" (UID: "bb56e6ff-b2bd-4186-8020-68cc37d41447"). InnerVolumeSpecName "kube-api-access-7mt4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:04:25.889904 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:25.889869 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mt4m\" (UniqueName: \"kubernetes.io/projected/bb56e6ff-b2bd-4186-8020-68cc37d41447-kube-api-access-7mt4m\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:26.463498 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.463414 2582 generic.go:356] "Generic (PLEG): container finished" podID="bb56e6ff-b2bd-4186-8020-68cc37d41447" containerID="d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2" exitCode=0 Jul 27 17:04:26.463852 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.463523 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" Jul 27 17:04:26.463852 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.463558 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" event={"ID":"bb56e6ff-b2bd-4186-8020-68cc37d41447","Type":"ContainerDied","Data":"d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2"} Jul 27 17:04:26.463852 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.463612 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7b2hr" event={"ID":"bb56e6ff-b2bd-4186-8020-68cc37d41447","Type":"ContainerDied","Data":"3df1d9eb6c18edcd8b4ab4d1a29def75db20f818d4e225aa4ed1be8c6f874dfe"} Jul 27 17:04:26.463852 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.463639 2582 scope.go:117] "RemoveContainer" containerID="d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2" Jul 27 17:04:26.472788 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.472761 2582 scope.go:117] "RemoveContainer" containerID="d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2" Jul 27 17:04:26.473098 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:04:26.473075 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2\": container with ID starting with d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2 not found: ID does not exist" containerID="d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2" Jul 27 17:04:26.473205 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.473113 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2"} err="failed to get container status \"d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2\": rpc error: code = NotFound desc = could not find container \"d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2\": container with ID starting with d9a38162ad6e1220fe3622a2742ecca34bce73ce1ae47c6a6f4f25021db23fa2 not found: ID does not exist" Jul 27 17:04:26.484052 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.484024 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7b2hr"] Jul 27 17:04:26.488054 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:26.488028 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7b2hr"] Jul 27 17:04:28.198508 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:28.198473 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb56e6ff-b2bd-4186-8020-68cc37d41447" path="/var/lib/kubelet/pods/bb56e6ff-b2bd-4186-8020-68cc37d41447/volumes" Jul 27 17:04:32.631506 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:32.631468 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:32.631959 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:32.631530 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:32.653201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:32.653173 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:33.522062 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:33.522033 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-gh5mt" Jul 27 17:04:40.433300 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.433264 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln"] Jul 27 17:04:40.433703 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.433684 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb56e6ff-b2bd-4186-8020-68cc37d41447" containerName="registry-server" Jul 27 17:04:40.433703 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.433697 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb56e6ff-b2bd-4186-8020-68cc37d41447" containerName="registry-server" Jul 27 17:04:40.433774 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.433762 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb56e6ff-b2bd-4186-8020-68cc37d41447" containerName="registry-server" Jul 27 17:04:40.438330 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.438311 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.441225 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.441206 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qmxv5\"" Jul 27 17:04:40.445996 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.445748 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln"] Jul 27 17:04:40.524487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.524451 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.524664 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.524504 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shvvd\" (UniqueName: \"kubernetes.io/projected/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-kube-api-access-shvvd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.524664 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.524608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.526415 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.526393 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx"] Jul 27 17:04:40.529824 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.529807 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.539003 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.538979 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx"] Jul 27 17:04:40.625351 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625312 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z9t8\" (UniqueName: \"kubernetes.io/projected/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-kube-api-access-8z9t8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.625553 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.625553 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.625553 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.625553 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625507 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shvvd\" (UniqueName: \"kubernetes.io/projected/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-kube-api-access-shvvd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.625553 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625544 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.625830 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625812 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.625980 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.625958 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.634894 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.634868 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shvvd\" (UniqueName: \"kubernetes.io/projected/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-kube-api-access-shvvd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.726925 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.726841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z9t8\" (UniqueName: \"kubernetes.io/projected/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-kube-api-access-8z9t8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.726925 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.726881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.727107 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.727002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.727250 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.727230 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.727317 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.727302 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.735505 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.735480 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z9t8\" (UniqueName: \"kubernetes.io/projected/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-kube-api-access-8z9t8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.748467 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.748444 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:40.840717 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.840681 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:40.889844 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.889818 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln"] Jul 27 17:04:40.892509 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:40.892476 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c869b39_0ac6_45b6_9ff0_4d1e3eb35f04.slice/crio-30ecbef51f67b9231773154d972adbb406391179290cb8cd571c4abd7451153b WatchSource:0}: Error finding container 30ecbef51f67b9231773154d972adbb406391179290cb8cd571c4abd7451153b: Status 404 returned error can't find the container with id 30ecbef51f67b9231773154d972adbb406391179290cb8cd571c4abd7451153b Jul 27 17:04:40.936693 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.936626 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc"] Jul 27 17:04:40.942123 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.942104 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:40.948127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.948105 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc"] Jul 27 17:04:40.996087 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:40.996060 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx"] Jul 27 17:04:40.999326 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:40.999288 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4fc03f_d9d3_419a_b7ed_11b203d83b8a.slice/crio-5e689952a25c161963bf5ffce69fcb9983ac2dcbcff855321f56654f8d030fd4 WatchSource:0}: Error finding container 5e689952a25c161963bf5ffce69fcb9983ac2dcbcff855321f56654f8d030fd4: Status 404 returned error can't find the container with id 5e689952a25c161963bf5ffce69fcb9983ac2dcbcff855321f56654f8d030fd4 Jul 27 17:04:41.030806 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.030777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.030941 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.030817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.030941 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.030925 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rz5q\" (UniqueName: \"kubernetes.io/projected/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-kube-api-access-2rz5q\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.131429 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.131368 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.131429 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.131431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.131673 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.131485 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rz5q\" (UniqueName: \"kubernetes.io/projected/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-kube-api-access-2rz5q\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.131930 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.131905 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.132008 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.131904 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.141277 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.141244 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rz5q\" (UniqueName: \"kubernetes.io/projected/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-kube-api-access-2rz5q\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.255529 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.255435 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:41.337956 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.337924 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb"] Jul 27 17:04:41.343714 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.343691 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.353018 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.352553 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb"] Jul 27 17:04:41.402829 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.402799 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc"] Jul 27 17:04:41.404483 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:41.404456 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebcb16b_0c9a_45df_b6fc_1c410fa6161e.slice/crio-64597c5ba25003b617dd73292d25fa5eba7121987fa7353631246d5d884d70d4 WatchSource:0}: Error finding container 64597c5ba25003b617dd73292d25fa5eba7121987fa7353631246d5d884d70d4: Status 404 returned error can't find the container with id 64597c5ba25003b617dd73292d25fa5eba7121987fa7353631246d5d884d70d4 Jul 27 17:04:41.434799 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.434769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.435142 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.434811 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggzn\" (UniqueName: \"kubernetes.io/projected/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-kube-api-access-fggzn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.435142 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.434847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.536338 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.536248 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.536338 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.536299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggzn\" (UniqueName: \"kubernetes.io/projected/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-kube-api-access-fggzn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.536600 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.536360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.536723 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.536687 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.536787 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.536734 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.542097 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.542058 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerID="364532e11bfc7a3a53e46d957e0bc2d8a9fae1f839439d5053937e3648ee762f" exitCode=0 Jul 27 17:04:41.542234 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.542150 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" event={"ID":"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e","Type":"ContainerDied","Data":"364532e11bfc7a3a53e46d957e0bc2d8a9fae1f839439d5053937e3648ee762f"} Jul 27 17:04:41.542234 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.542212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" event={"ID":"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e","Type":"ContainerStarted","Data":"64597c5ba25003b617dd73292d25fa5eba7121987fa7353631246d5d884d70d4"} Jul 27 17:04:41.543891 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.543871 2582 generic.go:356] "Generic (PLEG): container finished" podID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerID="891b9f1dfdfa2feceec1575189843a1de245212be1d502830bf1161ae4ae815c" exitCode=0 Jul 27 17:04:41.543975 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.543902 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" event={"ID":"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a","Type":"ContainerDied","Data":"891b9f1dfdfa2feceec1575189843a1de245212be1d502830bf1161ae4ae815c"} Jul 27 17:04:41.543975 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.543946 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" event={"ID":"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a","Type":"ContainerStarted","Data":"5e689952a25c161963bf5ffce69fcb9983ac2dcbcff855321f56654f8d030fd4"} Jul 27 17:04:41.545934 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.545886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggzn\" (UniqueName: \"kubernetes.io/projected/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-kube-api-access-fggzn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.545934 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.545903 2582 generic.go:356] "Generic (PLEG): container finished" podID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerID="72960ac80f79c295932fb5da6bfe59a2f585da349baa7f48e2b5ca6d2983b9e3" exitCode=0 Jul 27 17:04:41.546082 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.545976 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" event={"ID":"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04","Type":"ContainerDied","Data":"72960ac80f79c295932fb5da6bfe59a2f585da349baa7f48e2b5ca6d2983b9e3"} Jul 27 17:04:41.546082 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.546009 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" event={"ID":"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04","Type":"ContainerStarted","Data":"30ecbef51f67b9231773154d972adbb406391179290cb8cd571c4abd7451153b"} Jul 27 17:04:41.659704 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.659668 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:41.797977 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:41.797949 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb"] Jul 27 17:04:41.799779 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:41.799754 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa03a0ee_2a5a_48d5_86be_1e70e1ec4791.slice/crio-58045137f53c60835f3f4c5418526a615ba882a4311c019d285f1de009b3c5b5 WatchSource:0}: Error finding container 58045137f53c60835f3f4c5418526a615ba882a4311c019d285f1de009b3c5b5: Status 404 returned error can't find the container with id 58045137f53c60835f3f4c5418526a615ba882a4311c019d285f1de009b3c5b5 Jul 27 17:04:42.553752 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.553725 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerID="692d79d76dc4e78fe1c02a7e860146394bdafe9ab2602da3d85f659d78694e7f" exitCode=0 Jul 27 17:04:42.554134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.553814 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" event={"ID":"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791","Type":"ContainerDied","Data":"692d79d76dc4e78fe1c02a7e860146394bdafe9ab2602da3d85f659d78694e7f"} Jul 27 17:04:42.554134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.553882 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" event={"ID":"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791","Type":"ContainerStarted","Data":"58045137f53c60835f3f4c5418526a615ba882a4311c019d285f1de009b3c5b5"} Jul 27 17:04:42.555912 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.555846 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" event={"ID":"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e","Type":"ContainerStarted","Data":"7d254e671d525f065133638489859c2d5228e8ae66d8f65ba245d4b1554a1071"} Jul 27 17:04:42.558189 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.558156 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" event={"ID":"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a","Type":"ContainerStarted","Data":"58feb4287c58c6e7b02300a65eb5c04181e6c1526b4f3fe40f8c3618860cf74a"} Jul 27 17:04:42.560322 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.560299 2582 generic.go:356] "Generic (PLEG): container finished" podID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerID="26a37324ec9cc5dbf354ca9605c196c767d48ce5c40c5a297fb2780891be0fce" exitCode=0 Jul 27 17:04:42.560423 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:42.560349 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" event={"ID":"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04","Type":"ContainerDied","Data":"26a37324ec9cc5dbf354ca9605c196c767d48ce5c40c5a297fb2780891be0fce"} Jul 27 17:04:43.567894 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.567861 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerID="4bcc0a6f781e3996b47a03b61eae8f6558b56d19548d4ac015ab10d261c3ea1c" exitCode=0 Jul 27 17:04:43.568350 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.567936 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" event={"ID":"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791","Type":"ContainerDied","Data":"4bcc0a6f781e3996b47a03b61eae8f6558b56d19548d4ac015ab10d261c3ea1c"} Jul 27 17:04:43.569888 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.569864 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerID="7d254e671d525f065133638489859c2d5228e8ae66d8f65ba245d4b1554a1071" exitCode=0 Jul 27 17:04:43.569986 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.569913 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" event={"ID":"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e","Type":"ContainerDied","Data":"7d254e671d525f065133638489859c2d5228e8ae66d8f65ba245d4b1554a1071"} Jul 27 17:04:43.572019 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.571979 2582 generic.go:356] "Generic (PLEG): container finished" podID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerID="58feb4287c58c6e7b02300a65eb5c04181e6c1526b4f3fe40f8c3618860cf74a" exitCode=0 Jul 27 17:04:43.572196 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.572056 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" event={"ID":"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a","Type":"ContainerDied","Data":"58feb4287c58c6e7b02300a65eb5c04181e6c1526b4f3fe40f8c3618860cf74a"} Jul 27 17:04:43.581204 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.581174 2582 generic.go:356] "Generic (PLEG): container finished" podID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerID="60dac025f23c92245ae41272d6df4f2436bef8005700f3838a8d4f7512467b60" exitCode=0 Jul 27 17:04:43.581320 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:43.581214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" event={"ID":"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04","Type":"ContainerDied","Data":"60dac025f23c92245ae41272d6df4f2436bef8005700f3838a8d4f7512467b60"} Jul 27 17:04:44.588803 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.588760 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerID="7b53d1d5b9b73ad2e7531196d5e7cf4aa1fd10b477187d3cf72ed0d1e978371c" exitCode=0 Jul 27 17:04:44.589172 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.588845 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" event={"ID":"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791","Type":"ContainerDied","Data":"7b53d1d5b9b73ad2e7531196d5e7cf4aa1fd10b477187d3cf72ed0d1e978371c"} Jul 27 17:04:44.590895 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.590869 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerID="50874a876dcf2a751b83353aad172b0a65b106c61feec52024545ca0f9812e80" exitCode=0 Jul 27 17:04:44.591012 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.590945 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" event={"ID":"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e","Type":"ContainerDied","Data":"50874a876dcf2a751b83353aad172b0a65b106c61feec52024545ca0f9812e80"} Jul 27 17:04:44.592995 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.592973 2582 generic.go:356] "Generic (PLEG): container finished" podID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerID="63f1487efcf87b10d7f2ec409dba7492a2876c98bd1ce16da183a45bddfef608" exitCode=0 Jul 27 17:04:44.593078 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.593045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" event={"ID":"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a","Type":"ContainerDied","Data":"63f1487efcf87b10d7f2ec409dba7492a2876c98bd1ce16da183a45bddfef608"} Jul 27 17:04:44.728392 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.728354 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:44.853837 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.853761 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66cf8c55bd-2swwx"] Jul 27 17:04:44.854143 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854132 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="util" Jul 27 17:04:44.854184 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854147 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="util" Jul 27 17:04:44.854184 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854163 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="pull" Jul 27 17:04:44.854184 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854169 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="pull" Jul 27 17:04:44.854275 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854186 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="extract" Jul 27 17:04:44.854275 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854192 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="extract" Jul 27 17:04:44.854275 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.854256 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" containerName="extract" Jul 27 17:04:44.857249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.857220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.867144 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.867114 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66cf8c55bd-2swwx"] Jul 27 17:04:44.870174 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.870151 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-util\") pod \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " Jul 27 17:04:44.870283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.870249 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shvvd\" (UniqueName: \"kubernetes.io/projected/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-kube-api-access-shvvd\") pod \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " Jul 27 17:04:44.870351 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.870340 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-bundle\") pod \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\" (UID: \"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04\") " Jul 27 17:04:44.870862 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.870819 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-bundle" (OuterVolumeSpecName: "bundle") pod "6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" (UID: "6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:44.872327 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.872303 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-kube-api-access-shvvd" (OuterVolumeSpecName: "kube-api-access-shvvd") pod "6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" (UID: "6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04"). InnerVolumeSpecName "kube-api-access-shvvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:04:44.879336 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.879308 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-util" (OuterVolumeSpecName: "util") pod "6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04" (UID: "6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:44.971532 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2462602f-96e0-4173-95e6-656585f9d5e5-console-serving-cert\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971532 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971531 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2462602f-96e0-4173-95e6-656585f9d5e5-console-oauth-config\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-trusted-ca-bundle\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-console-config\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-service-ca\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971688 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-oauth-serving-cert\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971869 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971761 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx2nm\" (UniqueName: \"kubernetes.io/projected/2462602f-96e0-4173-95e6-656585f9d5e5-kube-api-access-nx2nm\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:44.971912 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971875 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:44.971912 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971887 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shvvd\" (UniqueName: \"kubernetes.io/projected/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-kube-api-access-shvvd\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:44.971912 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:44.971896 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.073074 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-console-config\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073074 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-service-ca\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073289 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073095 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-oauth-serving-cert\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073289 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx2nm\" (UniqueName: \"kubernetes.io/projected/2462602f-96e0-4173-95e6-656585f9d5e5-kube-api-access-nx2nm\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073289 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073172 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2462602f-96e0-4173-95e6-656585f9d5e5-console-serving-cert\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073289 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073186 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2462602f-96e0-4173-95e6-656585f9d5e5-console-oauth-config\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073289 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-trusted-ca-bundle\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073992 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073951 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-console-config\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073992 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-oauth-serving-cert\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.073992 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.073963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-service-ca\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.074239 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.074031 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2462602f-96e0-4173-95e6-656585f9d5e5-trusted-ca-bundle\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.075696 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.075679 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2462602f-96e0-4173-95e6-656585f9d5e5-console-serving-cert\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.075776 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.075743 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2462602f-96e0-4173-95e6-656585f9d5e5-console-oauth-config\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.082594 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.082575 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx2nm\" (UniqueName: \"kubernetes.io/projected/2462602f-96e0-4173-95e6-656585f9d5e5-kube-api-access-nx2nm\") pod \"console-66cf8c55bd-2swwx\" (UID: \"2462602f-96e0-4173-95e6-656585f9d5e5\") " pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.166774 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.166677 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:45.305399 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.305357 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66cf8c55bd-2swwx"] Jul 27 17:04:45.307177 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:45.307155 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2462602f_96e0_4173_95e6_656585f9d5e5.slice/crio-57d3a5069c1177ee325f4ab38b0c474ecbfb9ca3cfdbc2595b983a3783e12ded WatchSource:0}: Error finding container 57d3a5069c1177ee325f4ab38b0c474ecbfb9ca3cfdbc2595b983a3783e12ded: Status 404 returned error can't find the container with id 57d3a5069c1177ee325f4ab38b0c474ecbfb9ca3cfdbc2595b983a3783e12ded Jul 27 17:04:45.601084 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.601051 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" Jul 27 17:04:45.601526 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.601053 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln" event={"ID":"6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04","Type":"ContainerDied","Data":"30ecbef51f67b9231773154d972adbb406391179290cb8cd571c4abd7451153b"} Jul 27 17:04:45.601526 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.601181 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30ecbef51f67b9231773154d972adbb406391179290cb8cd571c4abd7451153b" Jul 27 17:04:45.606763 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.603866 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66cf8c55bd-2swwx" event={"ID":"2462602f-96e0-4173-95e6-656585f9d5e5","Type":"ContainerStarted","Data":"ab3c052721a0957ca0e3894aca6487858f5cdc87147736f0637802a84b93e393"} Jul 27 17:04:45.606763 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.603951 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66cf8c55bd-2swwx" event={"ID":"2462602f-96e0-4173-95e6-656585f9d5e5","Type":"ContainerStarted","Data":"57d3a5069c1177ee325f4ab38b0c474ecbfb9ca3cfdbc2595b983a3783e12ded"} Jul 27 17:04:45.633883 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.633609 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66cf8c55bd-2swwx" podStartSLOduration=1.633592169 podStartE2EDuration="1.633592169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:04:45.63155811 +0000 UTC m=+491.915717389" watchObservedRunningTime="2026-07-27 17:04:45.633592169 +0000 UTC m=+491.917751452" Jul 27 17:04:45.788888 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.788861 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:45.805402 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.805354 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:45.811865 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.811841 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:45.881078 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881000 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z9t8\" (UniqueName: \"kubernetes.io/projected/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-kube-api-access-8z9t8\") pod \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " Jul 27 17:04:45.881078 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881042 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-util\") pod \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " Jul 27 17:04:45.881280 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881117 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-util\") pod \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " Jul 27 17:04:45.881280 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881142 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-bundle\") pod \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " Jul 27 17:04:45.881280 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881162 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rz5q\" (UniqueName: \"kubernetes.io/projected/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-kube-api-access-2rz5q\") pod \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\" (UID: \"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e\") " Jul 27 17:04:45.881280 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881265 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-bundle\") pod \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\" (UID: \"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a\") " Jul 27 17:04:45.881819 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881788 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-bundle" (OuterVolumeSpecName: "bundle") pod "5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" (UID: "5b4fc03f-d9d3-419a-b7ed-11b203d83b8a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:45.881925 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.881799 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-bundle" (OuterVolumeSpecName: "bundle") pod "7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" (UID: "7ebcb16b-0c9a-45df-b6fc-1c410fa6161e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:45.883433 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.883406 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-kube-api-access-2rz5q" (OuterVolumeSpecName: "kube-api-access-2rz5q") pod "7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" (UID: "7ebcb16b-0c9a-45df-b6fc-1c410fa6161e"). InnerVolumeSpecName "kube-api-access-2rz5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:04:45.883634 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.883610 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-kube-api-access-8z9t8" (OuterVolumeSpecName: "kube-api-access-8z9t8") pod "5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" (UID: "5b4fc03f-d9d3-419a-b7ed-11b203d83b8a"). InnerVolumeSpecName "kube-api-access-8z9t8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:04:45.887179 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.887150 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-util" (OuterVolumeSpecName: "util") pod "5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" (UID: "5b4fc03f-d9d3-419a-b7ed-11b203d83b8a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:45.890582 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.890558 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-util" (OuterVolumeSpecName: "util") pod "7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" (UID: "7ebcb16b-0c9a-45df-b6fc-1c410fa6161e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:45.982804 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.982762 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-util\") pod \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " Jul 27 17:04:45.982964 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.982846 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fggzn\" (UniqueName: \"kubernetes.io/projected/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-kube-api-access-fggzn\") pod \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " Jul 27 17:04:45.982964 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.982878 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-bundle\") pod \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\" (UID: \"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791\") " Jul 27 17:04:45.983168 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983151 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z9t8\" (UniqueName: \"kubernetes.io/projected/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-kube-api-access-8z9t8\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.983503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983174 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.983503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983191 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.983503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983206 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.983503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983220 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rz5q\" (UniqueName: \"kubernetes.io/projected/7ebcb16b-0c9a-45df-b6fc-1c410fa6161e-kube-api-access-2rz5q\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.983503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983235 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b4fc03f-d9d3-419a-b7ed-11b203d83b8a-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:45.983716 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.983510 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-bundle" (OuterVolumeSpecName: "bundle") pod "fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" (UID: "fa03a0ee-2a5a-48d5-86be-1e70e1ec4791"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:45.984908 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.984881 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-kube-api-access-fggzn" (OuterVolumeSpecName: "kube-api-access-fggzn") pod "fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" (UID: "fa03a0ee-2a5a-48d5-86be-1e70e1ec4791"). InnerVolumeSpecName "kube-api-access-fggzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:04:45.988469 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:45.988446 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-util" (OuterVolumeSpecName: "util") pod "fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" (UID: "fa03a0ee-2a5a-48d5-86be-1e70e1ec4791"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:04:46.084193 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.084155 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-util\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:46.084193 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.084185 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fggzn\" (UniqueName: \"kubernetes.io/projected/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-kube-api-access-fggzn\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:46.084193 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.084196 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa03a0ee-2a5a-48d5-86be-1e70e1ec4791-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:04:46.609996 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.609959 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" Jul 27 17:04:46.610425 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.609963 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb" event={"ID":"fa03a0ee-2a5a-48d5-86be-1e70e1ec4791","Type":"ContainerDied","Data":"58045137f53c60835f3f4c5418526a615ba882a4311c019d285f1de009b3c5b5"} Jul 27 17:04:46.610425 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.610093 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58045137f53c60835f3f4c5418526a615ba882a4311c019d285f1de009b3c5b5" Jul 27 17:04:46.612094 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.611822 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" event={"ID":"7ebcb16b-0c9a-45df-b6fc-1c410fa6161e","Type":"ContainerDied","Data":"64597c5ba25003b617dd73292d25fa5eba7121987fa7353631246d5d884d70d4"} Jul 27 17:04:46.612094 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.611862 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64597c5ba25003b617dd73292d25fa5eba7121987fa7353631246d5d884d70d4" Jul 27 17:04:46.612094 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.611883 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc" Jul 27 17:04:46.613993 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.613976 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" Jul 27 17:04:46.614148 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.614014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx" event={"ID":"5b4fc03f-d9d3-419a-b7ed-11b203d83b8a","Type":"ContainerDied","Data":"5e689952a25c161963bf5ffce69fcb9983ac2dcbcff855321f56654f8d030fd4"} Jul 27 17:04:46.614148 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:46.614051 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e689952a25c161963bf5ffce69fcb9983ac2dcbcff855321f56654f8d030fd4" Jul 27 17:04:55.167350 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.167312 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:55.167883 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.167372 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:55.172119 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.172097 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:55.667814 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.667781 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66cf8c55bd-2swwx" Jul 27 17:04:55.716453 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.716418 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f55d6d855-t9wqs"] Jul 27 17:04:55.907549 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907508 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz"] Jul 27 17:04:55.907882 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907870 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="extract" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907884 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="extract" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907894 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="extract" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907900 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="extract" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907912 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="util" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907918 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="util" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907924 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="pull" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907930 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="pull" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907936 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="pull" Jul 27 17:04:55.907938 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907941 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="pull" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907951 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="extract" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907958 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="extract" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907969 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="util" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907973 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="util" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907983 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="util" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907988 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="util" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.907997 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="pull" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.908002 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="pull" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.908057 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa03a0ee-2a5a-48d5-86be-1e70e1ec4791" containerName="extract" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.908065 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b4fc03f-d9d3-419a-b7ed-11b203d83b8a" containerName="extract" Jul 27 17:04:55.908283 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.908071 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ebcb16b-0c9a-45df-b6fc-1c410fa6161e" containerName="extract" Jul 27 17:04:55.912500 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.912478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:04:55.919968 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.919948 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 27 17:04:55.920051 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.919950 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tggnj\"" Jul 27 17:04:55.925392 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:55.925354 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz"] Jul 27 17:04:56.077357 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:56.077307 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skdwc\" (UniqueName: \"kubernetes.io/projected/50a68335-f97e-4013-b568-544b66449076-kube-api-access-skdwc\") pod \"dns-operator-controller-manager-864dd78c4c-8lbcz\" (UID: \"50a68335-f97e-4013-b568-544b66449076\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:04:56.178876 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:56.178789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skdwc\" (UniqueName: \"kubernetes.io/projected/50a68335-f97e-4013-b568-544b66449076-kube-api-access-skdwc\") pod \"dns-operator-controller-manager-864dd78c4c-8lbcz\" (UID: \"50a68335-f97e-4013-b568-544b66449076\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:04:56.190281 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:56.190245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skdwc\" (UniqueName: \"kubernetes.io/projected/50a68335-f97e-4013-b568-544b66449076-kube-api-access-skdwc\") pod \"dns-operator-controller-manager-864dd78c4c-8lbcz\" (UID: \"50a68335-f97e-4013-b568-544b66449076\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:04:56.222808 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:56.222769 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:04:56.367851 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:56.367828 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz"] Jul 27 17:04:56.369510 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:04:56.369477 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a68335_f97e_4013_b568_544b66449076.slice/crio-46e5ff0c932333a8699d3ddcbf0e75ff2643598e466daecc04160dfa314689d4 WatchSource:0}: Error finding container 46e5ff0c932333a8699d3ddcbf0e75ff2643598e466daecc04160dfa314689d4: Status 404 returned error can't find the container with id 46e5ff0c932333a8699d3ddcbf0e75ff2643598e466daecc04160dfa314689d4 Jul 27 17:04:56.669956 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:56.669919 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" event={"ID":"50a68335-f97e-4013-b568-544b66449076","Type":"ContainerStarted","Data":"46e5ff0c932333a8699d3ddcbf0e75ff2643598e466daecc04160dfa314689d4"} Jul 27 17:04:59.698544 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.698508 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" event={"ID":"50a68335-f97e-4013-b568-544b66449076","Type":"ContainerStarted","Data":"3cdfbc426e8736bfd790297ca3146b8305367e69bef2147020b999a38239943c"} Jul 27 17:04:59.698957 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.698617 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:04:59.720261 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.720213 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" podStartSLOduration=2.2761799160000002 podStartE2EDuration="4.720197762s" totalImagesPullingTime="2.444017846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:04:59.718324375 +0000 UTC m=+506.002483658" watchObservedRunningTime="2026-07-27 17:04:59.720197762 +0000 UTC m=+506.004357045" Jul 27 17:04:59.978469 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.978385 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7"] Jul 27 17:04:59.981950 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.981934 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:04:59.985002 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.984983 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-h9fz7\"" Jul 27 17:04:59.992939 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:04:59.992916 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7"] Jul 27 17:05:00.013485 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:00.013457 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzs2s\" (UniqueName: \"kubernetes.io/projected/2add960a-7ba7-4222-8f23-1b59be175e99-kube-api-access-nzs2s\") pod \"authorino-operator-75cc8d5cb7-cwcr7\" (UID: \"2add960a-7ba7-4222-8f23-1b59be175e99\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:05:00.114671 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:00.114635 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzs2s\" (UniqueName: \"kubernetes.io/projected/2add960a-7ba7-4222-8f23-1b59be175e99-kube-api-access-nzs2s\") pod \"authorino-operator-75cc8d5cb7-cwcr7\" (UID: \"2add960a-7ba7-4222-8f23-1b59be175e99\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:05:00.128549 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:00.128519 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzs2s\" (UniqueName: \"kubernetes.io/projected/2add960a-7ba7-4222-8f23-1b59be175e99-kube-api-access-nzs2s\") pod \"authorino-operator-75cc8d5cb7-cwcr7\" (UID: \"2add960a-7ba7-4222-8f23-1b59be175e99\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:05:00.293076 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:00.292979 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:05:00.433567 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:00.433539 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7"] Jul 27 17:05:00.434839 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:05:00.434809 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2add960a_7ba7_4222_8f23_1b59be175e99.slice/crio-eaa76eead4a6665ebd05af2c18539660d6247316b88797f399161407548e2b27 WatchSource:0}: Error finding container eaa76eead4a6665ebd05af2c18539660d6247316b88797f399161407548e2b27: Status 404 returned error can't find the container with id eaa76eead4a6665ebd05af2c18539660d6247316b88797f399161407548e2b27 Jul 27 17:05:00.705286 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:00.705252 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" event={"ID":"2add960a-7ba7-4222-8f23-1b59be175e99","Type":"ContainerStarted","Data":"eaa76eead4a6665ebd05af2c18539660d6247316b88797f399161407548e2b27"} Jul 27 17:05:03.391118 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.391081 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv"] Jul 27 17:05:03.394700 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.394681 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:03.398228 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.398207 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4vt9g\"" Jul 27 17:05:03.431471 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.431441 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv"] Jul 27 17:05:03.443758 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.443730 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqkcp\" (UniqueName: \"kubernetes.io/projected/f09f9d07-42d1-4fcd-852d-84ecb4797534-kube-api-access-lqkcp\") pod \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" (UID: \"f09f9d07-42d1-4fcd-852d-84ecb4797534\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:03.544580 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.544545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqkcp\" (UniqueName: \"kubernetes.io/projected/f09f9d07-42d1-4fcd-852d-84ecb4797534-kube-api-access-lqkcp\") pod \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" (UID: \"f09f9d07-42d1-4fcd-852d-84ecb4797534\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:03.554896 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.554867 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqkcp\" (UniqueName: \"kubernetes.io/projected/f09f9d07-42d1-4fcd-852d-84ecb4797534-kube-api-access-lqkcp\") pod \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" (UID: \"f09f9d07-42d1-4fcd-852d-84ecb4797534\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:03.705179 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.705089 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:03.729932 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.729892 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" event={"ID":"2add960a-7ba7-4222-8f23-1b59be175e99","Type":"ContainerStarted","Data":"a97865e4d7cc79eb90febd85ff833e14ef7b1fa2506d15ad01bca4240c6a8f0c"} Jul 27 17:05:03.730064 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.730047 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:05:03.752001 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.751953 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" podStartSLOduration=2.340882657 podStartE2EDuration="4.751937078s" totalImagesPullingTime="2.411054421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:05:03.75015992 +0000 UTC m=+510.034319201" watchObservedRunningTime="2026-07-27 17:05:03.751937078 +0000 UTC m=+510.036096361" Jul 27 17:05:03.849729 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:03.849703 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv"] Jul 27 17:05:03.851032 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:05:03.851006 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09f9d07_42d1_4fcd_852d_84ecb4797534.slice/crio-9c7803682f18320991ea85018d4fc93f805a376023606d950e5171b7276bb79c WatchSource:0}: Error finding container 9c7803682f18320991ea85018d4fc93f805a376023606d950e5171b7276bb79c: Status 404 returned error can't find the container with id 9c7803682f18320991ea85018d4fc93f805a376023606d950e5171b7276bb79c Jul 27 17:05:04.737822 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:04.737779 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" event={"ID":"f09f9d07-42d1-4fcd-852d-84ecb4797534","Type":"ContainerStarted","Data":"9c7803682f18320991ea85018d4fc93f805a376023606d950e5171b7276bb79c"} Jul 27 17:05:06.751153 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:06.751110 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" event={"ID":"f09f9d07-42d1-4fcd-852d-84ecb4797534","Type":"ContainerStarted","Data":"2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9"} Jul 27 17:05:06.751619 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:06.751243 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:06.772338 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:06.772289 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" podStartSLOduration=1.741399617 podStartE2EDuration="3.772275566s" totalImagesPullingTime="2.030875949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:05:06.770207389 +0000 UTC m=+513.054366671" watchObservedRunningTime="2026-07-27 17:05:06.772275566 +0000 UTC m=+513.056434849" Jul 27 17:05:10.708374 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:10.708340 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8lbcz" Jul 27 17:05:14.740790 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:14.740756 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-cwcr7" Jul 27 17:05:17.758953 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.758921 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:17.902192 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.902162 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9b454"] Jul 27 17:05:17.907315 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.907292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:17.910625 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.910594 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 27 17:05:17.911143 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.911124 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 27 17:05:17.911515 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.911494 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qmxv5\"" Jul 27 17:05:17.918358 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.918332 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9b454"] Jul 27 17:05:17.984197 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.984164 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/591afeac-d683-4f16-97e4-c59fa0c740cb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:17.984365 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.984203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b67w6\" (UniqueName: \"kubernetes.io/projected/591afeac-d683-4f16-97e4-c59fa0c740cb-kube-api-access-b67w6\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:17.984365 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:17.984351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/591afeac-d683-4f16-97e4-c59fa0c740cb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.085669 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.085640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/591afeac-d683-4f16-97e4-c59fa0c740cb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.085872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.085699 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/591afeac-d683-4f16-97e4-c59fa0c740cb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.085872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.085730 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b67w6\" (UniqueName: \"kubernetes.io/projected/591afeac-d683-4f16-97e4-c59fa0c740cb-kube-api-access-b67w6\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.086291 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.086271 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/591afeac-d683-4f16-97e4-c59fa0c740cb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.088260 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.088239 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/591afeac-d683-4f16-97e4-c59fa0c740cb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.097057 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.097030 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b67w6\" (UniqueName: \"kubernetes.io/projected/591afeac-d683-4f16-97e4-c59fa0c740cb-kube-api-access-b67w6\") pod \"kuadrant-console-plugin-66b5774985-9b454\" (UID: \"591afeac-d683-4f16-97e4-c59fa0c740cb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.216922 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.216889 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" Jul 27 17:05:18.357649 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.357622 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9b454"] Jul 27 17:05:18.359873 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:05:18.359835 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod591afeac_d683_4f16_97e4_c59fa0c740cb.slice/crio-b0324dbc8e24c46a5c88e2ed8de978aeeff3381b4f564c793a06f8367c7deab1 WatchSource:0}: Error finding container b0324dbc8e24c46a5c88e2ed8de978aeeff3381b4f564c793a06f8367c7deab1: Status 404 returned error can't find the container with id b0324dbc8e24c46a5c88e2ed8de978aeeff3381b4f564c793a06f8367c7deab1 Jul 27 17:05:18.817911 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:18.817823 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" event={"ID":"591afeac-d683-4f16-97e4-c59fa0c740cb","Type":"ContainerStarted","Data":"b0324dbc8e24c46a5c88e2ed8de978aeeff3381b4f564c793a06f8367c7deab1"} Jul 27 17:05:20.742598 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:20.742418 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f55d6d855-t9wqs" podUID="1e2b7c21-1b8f-41cb-89dd-b730512f56e4" containerName="console" containerID="cri-o://f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746" gracePeriod=15 Jul 27 17:05:21.019822 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.019788 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:05:21.119741 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.119705 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-config\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.119929 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.119778 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghjc7\" (UniqueName: \"kubernetes.io/projected/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-kube-api-access-ghjc7\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.119929 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.119840 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-trusted-ca-bundle\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.119929 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.119892 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-service-ca\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.120104 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.119929 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-oauth-config\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.120104 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.119954 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-oauth-serving-cert\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.120104 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.120019 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-serving-cert\") pod \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\" (UID: \"1e2b7c21-1b8f-41cb-89dd-b730512f56e4\") " Jul 27 17:05:21.120252 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.120091 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-config" (OuterVolumeSpecName: "console-config") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:05:21.120419 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.120366 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:05:21.120552 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.120439 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.120631 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.120612 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-service-ca" (OuterVolumeSpecName: "service-ca") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:05:21.120937 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.120805 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:05:21.122867 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.122839 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:05:21.123307 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.123271 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:05:21.123428 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.123312 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-kube-api-access-ghjc7" (OuterVolumeSpecName: "kube-api-access-ghjc7") pod "1e2b7c21-1b8f-41cb-89dd-b730512f56e4" (UID: "1e2b7c21-1b8f-41cb-89dd-b730512f56e4"). InnerVolumeSpecName "kube-api-access-ghjc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:05:21.221947 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.221914 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.221947 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.221947 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-oauth-config\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.221947 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.221962 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-oauth-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.222256 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.221975 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-console-serving-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.222256 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.221989 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghjc7\" (UniqueName: \"kubernetes.io/projected/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-kube-api-access-ghjc7\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.222256 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.222002 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e2b7c21-1b8f-41cb-89dd-b730512f56e4-trusted-ca-bundle\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:21.840850 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.840817 2582 generic.go:356] "Generic (PLEG): container finished" podID="1e2b7c21-1b8f-41cb-89dd-b730512f56e4" containerID="f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746" exitCode=0 Jul 27 17:05:21.841341 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.840879 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f55d6d855-t9wqs" Jul 27 17:05:21.841341 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.840916 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f55d6d855-t9wqs" event={"ID":"1e2b7c21-1b8f-41cb-89dd-b730512f56e4","Type":"ContainerDied","Data":"f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746"} Jul 27 17:05:21.841341 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.841031 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f55d6d855-t9wqs" event={"ID":"1e2b7c21-1b8f-41cb-89dd-b730512f56e4","Type":"ContainerDied","Data":"b522c8ea66cf2fe5b046eb31f01b7c4b03a85fce9c3ae67d319870590a4b62b2"} Jul 27 17:05:21.841341 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.841090 2582 scope.go:117] "RemoveContainer" containerID="f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746" Jul 27 17:05:21.855802 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.854673 2582 scope.go:117] "RemoveContainer" containerID="f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746" Jul 27 17:05:21.855802 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:05:21.855104 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746\": container with ID starting with f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746 not found: ID does not exist" containerID="f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746" Jul 27 17:05:21.855802 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.855139 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746"} err="failed to get container status \"f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746\": rpc error: code = NotFound desc = could not find container \"f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746\": container with ID starting with f2082247dc526b34f5c33c95370ae2d7ec26c36f40f51297dbf199f77e5a3746 not found: ID does not exist" Jul 27 17:05:21.871983 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.871943 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f55d6d855-t9wqs"] Jul 27 17:05:21.875910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:21.875876 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f55d6d855-t9wqs"] Jul 27 17:05:22.200994 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:22.200910 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e2b7c21-1b8f-41cb-89dd-b730512f56e4" path="/var/lib/kubelet/pods/1e2b7c21-1b8f-41cb-89dd-b730512f56e4/volumes" Jul 27 17:05:28.598095 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.598062 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv"] Jul 27 17:05:28.598513 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.598331 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" containerName="manager" containerID="cri-o://2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9" gracePeriod=2 Jul 27 17:05:28.606472 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.606135 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv"] Jul 27 17:05:28.627400 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.627358 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p"] Jul 27 17:05:28.628117 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.628089 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" containerName="manager" Jul 27 17:05:28.628117 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.628117 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" containerName="manager" Jul 27 17:05:28.628249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.628147 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e2b7c21-1b8f-41cb-89dd-b730512f56e4" containerName="console" Jul 27 17:05:28.628249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.628155 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e2b7c21-1b8f-41cb-89dd-b730512f56e4" containerName="console" Jul 27 17:05:28.628321 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.628250 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" containerName="manager" Jul 27 17:05:28.628321 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.628265 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e2b7c21-1b8f-41cb-89dd-b730512f56e4" containerName="console" Jul 27 17:05:28.632325 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.631824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:05:28.644275 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.644245 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p"] Jul 27 17:05:28.662954 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:05:28.662911 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:28.806608 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.806565 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvk5n\" (UniqueName: \"kubernetes.io/projected/66b1081e-5294-42d5-8378-8f023cdb4fc3-kube-api-access-lvk5n\") pod \"limitador-operator-controller-manager-859d7b78bd-qdl6p\" (UID: \"66b1081e-5294-42d5-8378-8f023cdb4fc3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:05:28.908200 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.908109 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvk5n\" (UniqueName: \"kubernetes.io/projected/66b1081e-5294-42d5-8378-8f023cdb4fc3-kube-api-access-lvk5n\") pod \"limitador-operator-controller-manager-859d7b78bd-qdl6p\" (UID: \"66b1081e-5294-42d5-8378-8f023cdb4fc3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:05:28.927265 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.927234 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvk5n\" (UniqueName: \"kubernetes.io/projected/66b1081e-5294-42d5-8378-8f023cdb4fc3-kube-api-access-lvk5n\") pod \"limitador-operator-controller-manager-859d7b78bd-qdl6p\" (UID: \"66b1081e-5294-42d5-8378-8f023cdb4fc3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:05:28.985464 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:28.985427 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:05:40.252744 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.250320 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p"] Jul 27 17:05:40.255862 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:05:40.255827 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b1081e_5294_42d5_8378_8f023cdb4fc3.slice/crio-a0deeed445d7d6555f2e88ae3008dd9d8687a6cd0faff057e917cf36a755275b WatchSource:0}: Error finding container a0deeed445d7d6555f2e88ae3008dd9d8687a6cd0faff057e917cf36a755275b: Status 404 returned error can't find the container with id a0deeed445d7d6555f2e88ae3008dd9d8687a6cd0faff057e917cf36a755275b Jul 27 17:05:40.258874 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.258857 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:40.262285 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:05:40.262256 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:40.323217 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.323188 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqkcp\" (UniqueName: \"kubernetes.io/projected/f09f9d07-42d1-4fcd-852d-84ecb4797534-kube-api-access-lqkcp\") pod \"f09f9d07-42d1-4fcd-852d-84ecb4797534\" (UID: \"f09f9d07-42d1-4fcd-852d-84ecb4797534\") " Jul 27 17:05:40.325134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.325099 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f09f9d07-42d1-4fcd-852d-84ecb4797534-kube-api-access-lqkcp" (OuterVolumeSpecName: "kube-api-access-lqkcp") pod "f09f9d07-42d1-4fcd-852d-84ecb4797534" (UID: "f09f9d07-42d1-4fcd-852d-84ecb4797534"). InnerVolumeSpecName "kube-api-access-lqkcp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:05:40.424725 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.424648 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqkcp\" (UniqueName: \"kubernetes.io/projected/f09f9d07-42d1-4fcd-852d-84ecb4797534-kube-api-access-lqkcp\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:05:40.975202 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.975168 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" event={"ID":"66b1081e-5294-42d5-8378-8f023cdb4fc3","Type":"ContainerStarted","Data":"1c73ed03e263c116cd97ec916f33d757102f19de2624110400bbb34dbefb1806"} Jul 27 17:05:40.975434 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.975234 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:05:40.975434 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.975256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" event={"ID":"66b1081e-5294-42d5-8378-8f023cdb4fc3","Type":"ContainerStarted","Data":"a0deeed445d7d6555f2e88ae3008dd9d8687a6cd0faff057e917cf36a755275b"} Jul 27 17:05:40.976466 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.976441 2582 generic.go:356] "Generic (PLEG): container finished" podID="f09f9d07-42d1-4fcd-852d-84ecb4797534" containerID="2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9" exitCode=0 Jul 27 17:05:40.976594 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.976483 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:40.976594 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.976517 2582 scope.go:117] "RemoveContainer" containerID="2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9" Jul 27 17:05:40.981771 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.981742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" event={"ID":"591afeac-d683-4f16-97e4-c59fa0c740cb","Type":"ContainerStarted","Data":"cbdaa9e58b4ffb30d62e05d0fbab03125c25de5f8563a9719c10a371a45dd989"} Jul 27 17:05:40.990717 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.990697 2582 scope.go:117] "RemoveContainer" containerID="2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9" Jul 27 17:05:40.991006 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:05:40.990985 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9\": container with ID starting with 2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9 not found: ID does not exist" containerID="2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9" Jul 27 17:05:40.991075 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.991014 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9"} err="failed to get container status \"2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9\": rpc error: code = NotFound desc = could not find container \"2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9\": container with ID starting with 2df506795a783910b9fff2cefc696b45a06e5498bba41db5663042fa9a2ce7d9 not found: ID does not exist" Jul 27 17:05:40.996345 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:40.996301 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" podStartSLOduration=12.996285921 podStartE2EDuration="12.996285921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:05:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:05:40.994337526 +0000 UTC m=+547.278496808" watchObservedRunningTime="2026-07-27 17:05:40.996285921 +0000 UTC m=+547.280445203" Jul 27 17:05:40.996667 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:05:40.996643 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:40.998915 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:05:40.998890 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-fb9lv\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fb9lv" Jul 27 17:05:41.017641 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:41.017589 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9b454" podStartSLOduration=2.193533651 podStartE2EDuration="24.017573347s" totalImagesPullingTime="21.824039696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:05:41.015850908 +0000 UTC m=+547.300010182" watchObservedRunningTime="2026-07-27 17:05:41.017573347 +0000 UTC m=+547.301732629" Jul 27 17:05:42.199326 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:42.199293 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f09f9d07-42d1-4fcd-852d-84ecb4797534" path="/var/lib/kubelet/pods/f09f9d07-42d1-4fcd-852d-84ecb4797534/volumes" Jul 27 17:05:51.990770 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:05:51.990738 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdl6p" Jul 27 17:06:09.075306 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.075266 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-k8bhr"] Jul 27 17:06:09.219189 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.219157 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k8bhr"] Jul 27 17:06:09.219353 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.219263 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:09.222645 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.222618 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t2r6d\"" Jul 27 17:06:09.304202 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.304170 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wgxpj"] Jul 27 17:06:09.346538 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.346504 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wgxpj"] Jul 27 17:06:09.346711 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.346605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:09.395592 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.395557 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rws8p\" (UniqueName: \"kubernetes.io/projected/753ce14d-93f7-4fb8-9ff5-059146152a26-kube-api-access-rws8p\") pod \"authorino-58cd955675-k8bhr\" (UID: \"753ce14d-93f7-4fb8-9ff5-059146152a26\") " pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:09.496505 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.496470 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rws8p\" (UniqueName: \"kubernetes.io/projected/753ce14d-93f7-4fb8-9ff5-059146152a26-kube-api-access-rws8p\") pod \"authorino-58cd955675-k8bhr\" (UID: \"753ce14d-93f7-4fb8-9ff5-059146152a26\") " pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:09.496664 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.496568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjtjc\" (UniqueName: \"kubernetes.io/projected/98452c7c-ab21-4866-935b-a7b09031b2de-kube-api-access-cjtjc\") pod \"authorino-75d556bd77-wgxpj\" (UID: \"98452c7c-ab21-4866-935b-a7b09031b2de\") " pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:09.508432 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.508409 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rws8p\" (UniqueName: \"kubernetes.io/projected/753ce14d-93f7-4fb8-9ff5-059146152a26-kube-api-access-rws8p\") pod \"authorino-58cd955675-k8bhr\" (UID: \"753ce14d-93f7-4fb8-9ff5-059146152a26\") " pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:09.530086 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.530057 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:09.597288 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.597220 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjtjc\" (UniqueName: \"kubernetes.io/projected/98452c7c-ab21-4866-935b-a7b09031b2de-kube-api-access-cjtjc\") pod \"authorino-75d556bd77-wgxpj\" (UID: \"98452c7c-ab21-4866-935b-a7b09031b2de\") " pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:09.607456 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.607428 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjtjc\" (UniqueName: \"kubernetes.io/projected/98452c7c-ab21-4866-935b-a7b09031b2de-kube-api-access-cjtjc\") pod \"authorino-75d556bd77-wgxpj\" (UID: \"98452c7c-ab21-4866-935b-a7b09031b2de\") " pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:09.656605 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.656573 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:09.738902 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.738843 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-k8bhr"] Jul 27 17:06:09.741835 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:06:09.741805 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod753ce14d_93f7_4fb8_9ff5_059146152a26.slice/crio-e90980e97a62508625d22368f63ebdc903d10ae312a5fa059b8f9910f32cc199 WatchSource:0}: Error finding container e90980e97a62508625d22368f63ebdc903d10ae312a5fa059b8f9910f32cc199: Status 404 returned error can't find the container with id e90980e97a62508625d22368f63ebdc903d10ae312a5fa059b8f9910f32cc199 Jul 27 17:06:09.818783 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:09.818755 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wgxpj"] Jul 27 17:06:09.820144 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:06:09.820109 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98452c7c_ab21_4866_935b_a7b09031b2de.slice/crio-3a931e31e03b6363d2749172328ed108357c523e1f81a276e89964f05c56f8b5 WatchSource:0}: Error finding container 3a931e31e03b6363d2749172328ed108357c523e1f81a276e89964f05c56f8b5: Status 404 returned error can't find the container with id 3a931e31e03b6363d2749172328ed108357c523e1f81a276e89964f05c56f8b5 Jul 27 17:06:10.140057 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:10.140022 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wgxpj" event={"ID":"98452c7c-ab21-4866-935b-a7b09031b2de","Type":"ContainerStarted","Data":"3a931e31e03b6363d2749172328ed108357c523e1f81a276e89964f05c56f8b5"} Jul 27 17:06:10.141240 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:10.141216 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k8bhr" event={"ID":"753ce14d-93f7-4fb8-9ff5-059146152a26","Type":"ContainerStarted","Data":"e90980e97a62508625d22368f63ebdc903d10ae312a5fa059b8f9910f32cc199"} Jul 27 17:06:14.173886 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:14.173849 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wgxpj" event={"ID":"98452c7c-ab21-4866-935b-a7b09031b2de","Type":"ContainerStarted","Data":"f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0"} Jul 27 17:06:14.175357 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:14.175330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k8bhr" event={"ID":"753ce14d-93f7-4fb8-9ff5-059146152a26","Type":"ContainerStarted","Data":"6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184"} Jul 27 17:06:14.191956 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:14.191901 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wgxpj" podStartSLOduration=1.5654648880000002 podStartE2EDuration="5.19188674s" totalImagesPullingTime="3.626421852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:06:14.190984519 +0000 UTC m=+580.475143801" watchObservedRunningTime="2026-07-27 17:06:14.19188674 +0000 UTC m=+580.476046022" Jul 27 17:06:14.220688 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:14.220637 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-k8bhr" podStartSLOduration=1.503957528 podStartE2EDuration="5.220621474s" totalImagesPullingTime="3.716663946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:06:14.21748495 +0000 UTC m=+580.501644246" watchObservedRunningTime="2026-07-27 17:06:14.220621474 +0000 UTC m=+580.504780758" Jul 27 17:06:14.239845 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:14.239814 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k8bhr"] Jul 27 17:06:16.185815 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:16.185744 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-k8bhr" podUID="753ce14d-93f7-4fb8-9ff5-059146152a26" containerName="authorino" containerID="cri-o://6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184" gracePeriod=30 Jul 27 17:06:16.455161 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:16.455137 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:16.466683 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:16.466656 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rws8p\" (UniqueName: \"kubernetes.io/projected/753ce14d-93f7-4fb8-9ff5-059146152a26-kube-api-access-rws8p\") pod \"753ce14d-93f7-4fb8-9ff5-059146152a26\" (UID: \"753ce14d-93f7-4fb8-9ff5-059146152a26\") " Jul 27 17:06:16.468790 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:16.468763 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753ce14d-93f7-4fb8-9ff5-059146152a26-kube-api-access-rws8p" (OuterVolumeSpecName: "kube-api-access-rws8p") pod "753ce14d-93f7-4fb8-9ff5-059146152a26" (UID: "753ce14d-93f7-4fb8-9ff5-059146152a26"). InnerVolumeSpecName "kube-api-access-rws8p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:06:16.567277 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:16.567241 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rws8p\" (UniqueName: \"kubernetes.io/projected/753ce14d-93f7-4fb8-9ff5-059146152a26-kube-api-access-rws8p\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:06:17.192572 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.192536 2582 generic.go:356] "Generic (PLEG): container finished" podID="753ce14d-93f7-4fb8-9ff5-059146152a26" containerID="6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184" exitCode=0 Jul 27 17:06:17.192572 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.192574 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-k8bhr" Jul 27 17:06:17.193083 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.192591 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k8bhr" event={"ID":"753ce14d-93f7-4fb8-9ff5-059146152a26","Type":"ContainerDied","Data":"6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184"} Jul 27 17:06:17.193083 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.192641 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-k8bhr" event={"ID":"753ce14d-93f7-4fb8-9ff5-059146152a26","Type":"ContainerDied","Data":"e90980e97a62508625d22368f63ebdc903d10ae312a5fa059b8f9910f32cc199"} Jul 27 17:06:17.193083 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.192665 2582 scope.go:117] "RemoveContainer" containerID="6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184" Jul 27 17:06:17.201806 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.201790 2582 scope.go:117] "RemoveContainer" containerID="6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184" Jul 27 17:06:17.202057 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:06:17.202039 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184\": container with ID starting with 6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184 not found: ID does not exist" containerID="6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184" Jul 27 17:06:17.202108 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.202067 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184"} err="failed to get container status \"6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184\": rpc error: code = NotFound desc = could not find container \"6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184\": container with ID starting with 6172f9cca648c96705ea511220d218e2ce7c390f6d24243b4547494ce4afb184 not found: ID does not exist" Jul 27 17:06:17.216308 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.216281 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-k8bhr"] Jul 27 17:06:17.224333 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:17.224311 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-k8bhr"] Jul 27 17:06:18.198621 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:18.198578 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="753ce14d-93f7-4fb8-9ff5-059146152a26" path="/var/lib/kubelet/pods/753ce14d-93f7-4fb8-9ff5-059146152a26/volumes" Jul 27 17:06:46.015775 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.015729 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7rkg7"] Jul 27 17:06:46.016249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.016229 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="753ce14d-93f7-4fb8-9ff5-059146152a26" containerName="authorino" Jul 27 17:06:46.016249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.016244 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="753ce14d-93f7-4fb8-9ff5-059146152a26" containerName="authorino" Jul 27 17:06:46.016329 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.016324 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="753ce14d-93f7-4fb8-9ff5-059146152a26" containerName="authorino" Jul 27 17:06:46.024731 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.024705 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7rkg7"] Jul 27 17:06:46.024871 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.024806 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:46.147033 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.146993 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qht9v\" (UniqueName: \"kubernetes.io/projected/fb686a9b-20d8-409d-a953-c691812b670d-kube-api-access-qht9v\") pod \"authorino-5d4b67c446-7rkg7\" (UID: \"fb686a9b-20d8-409d-a953-c691812b670d\") " pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:46.245049 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.245006 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7rkg7"] Jul 27 17:06:46.245363 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:06:46.245337 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qht9v], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-7rkg7" podUID="fb686a9b-20d8-409d-a953-c691812b670d" Jul 27 17:06:46.247944 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.247919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qht9v\" (UniqueName: \"kubernetes.io/projected/fb686a9b-20d8-409d-a953-c691812b670d-kube-api-access-qht9v\") pod \"authorino-5d4b67c446-7rkg7\" (UID: \"fb686a9b-20d8-409d-a953-c691812b670d\") " pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:46.273111 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.273039 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qht9v\" (UniqueName: \"kubernetes.io/projected/fb686a9b-20d8-409d-a953-c691812b670d-kube-api-access-qht9v\") pod \"authorino-5d4b67c446-7rkg7\" (UID: \"fb686a9b-20d8-409d-a953-c691812b670d\") " pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:46.355817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.355789 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:46.364064 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.364032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:46.449583 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.449548 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qht9v\" (UniqueName: \"kubernetes.io/projected/fb686a9b-20d8-409d-a953-c691812b670d-kube-api-access-qht9v\") pod \"fb686a9b-20d8-409d-a953-c691812b670d\" (UID: \"fb686a9b-20d8-409d-a953-c691812b670d\") " Jul 27 17:06:46.451587 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.451564 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb686a9b-20d8-409d-a953-c691812b670d-kube-api-access-qht9v" (OuterVolumeSpecName: "kube-api-access-qht9v") pod "fb686a9b-20d8-409d-a953-c691812b670d" (UID: "fb686a9b-20d8-409d-a953-c691812b670d"). InnerVolumeSpecName "kube-api-access-qht9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:06:46.550736 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:46.550659 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qht9v\" (UniqueName: \"kubernetes.io/projected/fb686a9b-20d8-409d-a953-c691812b670d-kube-api-access-qht9v\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:06:47.360952 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:47.360919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7rkg7" Jul 27 17:06:47.420392 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:47.420349 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7rkg7"] Jul 27 17:06:47.433830 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:47.433801 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7rkg7"] Jul 27 17:06:48.201918 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:48.201885 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb686a9b-20d8-409d-a953-c691812b670d" path="/var/lib/kubelet/pods/fb686a9b-20d8-409d-a953-c691812b670d/volumes" Jul 27 17:06:50.325562 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:50.325527 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wgxpj"] Jul 27 17:06:50.325998 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:50.325725 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wgxpj" podUID="98452c7c-ab21-4866-935b-a7b09031b2de" containerName="authorino" containerID="cri-o://f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0" gracePeriod=30 Jul 27 17:06:50.585685 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:50.585661 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:50.686553 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:50.686436 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjtjc\" (UniqueName: \"kubernetes.io/projected/98452c7c-ab21-4866-935b-a7b09031b2de-kube-api-access-cjtjc\") pod \"98452c7c-ab21-4866-935b-a7b09031b2de\" (UID: \"98452c7c-ab21-4866-935b-a7b09031b2de\") " Jul 27 17:06:50.695834 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:50.688687 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98452c7c-ab21-4866-935b-a7b09031b2de-kube-api-access-cjtjc" (OuterVolumeSpecName: "kube-api-access-cjtjc") pod "98452c7c-ab21-4866-935b-a7b09031b2de" (UID: "98452c7c-ab21-4866-935b-a7b09031b2de"). InnerVolumeSpecName "kube-api-access-cjtjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:06:50.788053 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:50.787998 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjtjc\" (UniqueName: \"kubernetes.io/projected/98452c7c-ab21-4866-935b-a7b09031b2de-kube-api-access-cjtjc\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:06:51.384177 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.384133 2582 generic.go:356] "Generic (PLEG): container finished" podID="98452c7c-ab21-4866-935b-a7b09031b2de" containerID="f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0" exitCode=0 Jul 27 17:06:51.384177 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.384174 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wgxpj" Jul 27 17:06:51.384700 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.384212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wgxpj" event={"ID":"98452c7c-ab21-4866-935b-a7b09031b2de","Type":"ContainerDied","Data":"f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0"} Jul 27 17:06:51.384700 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.384261 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wgxpj" event={"ID":"98452c7c-ab21-4866-935b-a7b09031b2de","Type":"ContainerDied","Data":"3a931e31e03b6363d2749172328ed108357c523e1f81a276e89964f05c56f8b5"} Jul 27 17:06:51.384700 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.384288 2582 scope.go:117] "RemoveContainer" containerID="f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0" Jul 27 17:06:51.393761 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.393739 2582 scope.go:117] "RemoveContainer" containerID="f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0" Jul 27 17:06:51.394025 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:06:51.394004 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0\": container with ID starting with f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0 not found: ID does not exist" containerID="f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0" Jul 27 17:06:51.394124 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.394033 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0"} err="failed to get container status \"f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0\": rpc error: code = NotFound desc = could not find container \"f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0\": container with ID starting with f2465389ee298653aa68a8df0c70fe8fabe9adad5b18766e94058b0d710a15f0 not found: ID does not exist" Jul 27 17:06:51.446625 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.446585 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wgxpj"] Jul 27 17:06:51.456860 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:51.456832 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wgxpj"] Jul 27 17:06:52.198922 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:06:52.198880 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98452c7c-ab21-4866-935b-a7b09031b2de" path="/var/lib/kubelet/pods/98452c7c-ab21-4866-935b-a7b09031b2de/volumes" Jul 27 17:07:32.566370 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.566330 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57dc978488-w2jb7"] Jul 27 17:07:32.567530 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.567502 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98452c7c-ab21-4866-935b-a7b09031b2de" containerName="authorino" Jul 27 17:07:32.567530 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.567532 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="98452c7c-ab21-4866-935b-a7b09031b2de" containerName="authorino" Jul 27 17:07:32.567769 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.567755 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="98452c7c-ab21-4866-935b-a7b09031b2de" containerName="authorino" Jul 27 17:07:32.574865 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.574838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.578105 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.578085 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 27 17:07:32.579496 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.579470 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 27 17:07:32.579496 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.579488 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vcrrj\"" Jul 27 17:07:32.579698 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.579470 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 27 17:07:32.584551 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.584526 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57dc978488-w2jb7"] Jul 27 17:07:32.678472 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.678442 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aba2282-7079-4d59-9d46-ee74dce8cce7-maas-api-tls\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.678665 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.678507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgt8f\" (UniqueName: \"kubernetes.io/projected/7aba2282-7079-4d59-9d46-ee74dce8cce7-kube-api-access-fgt8f\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.678665 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.678601 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aba2282-7079-4d59-9d46-ee74dce8cce7-openshift-service-ca\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.779718 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.779688 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aba2282-7079-4d59-9d46-ee74dce8cce7-maas-api-tls\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.779910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.779756 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgt8f\" (UniqueName: \"kubernetes.io/projected/7aba2282-7079-4d59-9d46-ee74dce8cce7-kube-api-access-fgt8f\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.779910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.779808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aba2282-7079-4d59-9d46-ee74dce8cce7-openshift-service-ca\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.780455 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.780433 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7aba2282-7079-4d59-9d46-ee74dce8cce7-openshift-service-ca\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.782346 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.782313 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7aba2282-7079-4d59-9d46-ee74dce8cce7-maas-api-tls\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.790594 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.790567 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgt8f\" (UniqueName: \"kubernetes.io/projected/7aba2282-7079-4d59-9d46-ee74dce8cce7-kube-api-access-fgt8f\") pod \"maas-api-57dc978488-w2jb7\" (UID: \"7aba2282-7079-4d59-9d46-ee74dce8cce7\") " pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:32.888059 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:32.888023 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:33.049858 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:33.049831 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57dc978488-w2jb7"] Jul 27 17:07:33.052108 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:07:33.052080 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aba2282_7079_4d59_9d46_ee74dce8cce7.slice/crio-e91e1e52d96e5523724ee3a2cb31afb9dedcf3100c09a0c217edd98a7b6ae626 WatchSource:0}: Error finding container e91e1e52d96e5523724ee3a2cb31afb9dedcf3100c09a0c217edd98a7b6ae626: Status 404 returned error can't find the container with id e91e1e52d96e5523724ee3a2cb31afb9dedcf3100c09a0c217edd98a7b6ae626 Jul 27 17:07:33.053590 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:33.053568 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 17:07:33.612253 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:33.612216 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" event={"ID":"7aba2282-7079-4d59-9d46-ee74dce8cce7","Type":"ContainerStarted","Data":"e91e1e52d96e5523724ee3a2cb31afb9dedcf3100c09a0c217edd98a7b6ae626"} Jul 27 17:07:36.632598 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:36.632553 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" event={"ID":"7aba2282-7079-4d59-9d46-ee74dce8cce7","Type":"ContainerStarted","Data":"f3a2e2614bc98495e93de62e2fac237eb5760b61ca30156a12eefbce15f4f375"} Jul 27 17:07:36.633140 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:36.632655 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:07:36.665447 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:36.665370 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" podStartSLOduration=1.6304308060000001 podStartE2EDuration="4.665355504s" totalImagesPullingTime="3.034924698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:07:36.663221631 +0000 UTC m=+662.947380914" watchObservedRunningTime="2026-07-27 17:07:36.665355504 +0000 UTC m=+662.949514819" Jul 27 17:07:42.643888 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:07:42.643861 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57dc978488-w2jb7" Jul 27 17:08:11.228693 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.228647 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn"] Jul 27 17:08:11.233336 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.233312 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.236534 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.236512 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 27 17:08:11.237613 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.237593 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6x9nq\"" Jul 27 17:08:11.237734 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.237627 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 27 17:08:11.237734 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.237627 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 27 17:08:11.242980 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.242959 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn"] Jul 27 17:08:11.350567 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.350535 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.350745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.350592 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6v52\" (UniqueName: \"kubernetes.io/projected/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-kube-api-access-z6v52\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.350745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.350638 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.350745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.350723 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.350900 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.350822 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.350900 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.350871 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.452575 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.452532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.452766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.452600 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.452766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.452631 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6v52\" (UniqueName: \"kubernetes.io/projected/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-kube-api-access-z6v52\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.452766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.452654 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.452766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.452708 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.452766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.452743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.453173 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.453131 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.453173 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.453144 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.453329 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.453176 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.454806 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.454788 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.455211 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.455190 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.466337 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.466309 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6v52\" (UniqueName: \"kubernetes.io/projected/59d8a06c-e478-4ed8-920c-7e48dbbd1e9d-kube-api-access-z6v52\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn\" (UID: \"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.545373 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.545282 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:11.690759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.690718 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn"] Jul 27 17:08:11.694029 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:08:11.694003 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d8a06c_e478_4ed8_920c_7e48dbbd1e9d.slice/crio-63eb3f25beee3a7298538b5a05b86bef156e0ffbae061c9e5a703991af0bf42b WatchSource:0}: Error finding container 63eb3f25beee3a7298538b5a05b86bef156e0ffbae061c9e5a703991af0bf42b: Status 404 returned error can't find the container with id 63eb3f25beee3a7298538b5a05b86bef156e0ffbae061c9e5a703991af0bf42b Jul 27 17:08:11.831446 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:11.831351 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" event={"ID":"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d","Type":"ContainerStarted","Data":"63eb3f25beee3a7298538b5a05b86bef156e0ffbae061c9e5a703991af0bf42b"} Jul 27 17:08:12.838690 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:12.838646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" event={"ID":"59d8a06c-e478-4ed8-920c-7e48dbbd1e9d","Type":"ContainerStarted","Data":"b7796a294ff3f7d8b237c382f3c2330c46cf3010f927a1958e0dd2f0e33f1ecd"} Jul 27 17:08:12.871450 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:12.871398 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" podStartSLOduration=0.851990021 podStartE2EDuration="1.871358592s" totalImagesPullingTime="1.019368571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:08:12.866638473 +0000 UTC m=+699.150797740" watchObservedRunningTime="2026-07-27 17:08:12.871358592 +0000 UTC m=+699.155517900" Jul 27 17:08:21.546297 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:21.546252 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:21.546722 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:21.546323 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:21.558667 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:21.558639 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:08:21.903123 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:08:21.903086 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn" Jul 27 17:09:10.522088 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.522049 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5"] Jul 27 17:09:10.524775 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.524758 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.528284 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.528266 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 27 17:09:10.538089 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.538065 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5"] Jul 27 17:09:10.615883 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.615849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2x9b\" (UniqueName: \"kubernetes.io/projected/2926f79c-8d6d-4013-9953-bfe9d962d618-kube-api-access-s2x9b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.616060 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.615903 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.616060 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.615971 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.616060 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.616031 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.616201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.616093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2926f79c-8d6d-4013-9953-bfe9d962d618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.616201 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.616113 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.716699 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.716666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.716699 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.716709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.716945 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.716732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.716945 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.716783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2926f79c-8d6d-4013-9953-bfe9d962d618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.716945 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.716804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.716945 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.716857 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2x9b\" (UniqueName: \"kubernetes.io/projected/2926f79c-8d6d-4013-9953-bfe9d962d618-kube-api-access-s2x9b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.717164 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.717050 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.717267 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.717223 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.717267 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.717247 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.719068 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.719040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2926f79c-8d6d-4013-9953-bfe9d962d618-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.719193 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.719153 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2926f79c-8d6d-4013-9953-bfe9d962d618-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.725338 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.725314 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2x9b\" (UniqueName: \"kubernetes.io/projected/2926f79c-8d6d-4013-9953-bfe9d962d618-kube-api-access-s2x9b\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-47tk5\" (UID: \"2926f79c-8d6d-4013-9953-bfe9d962d618\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.835657 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.835624 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:10.972134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:10.972108 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5"] Jul 27 17:09:10.973098 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:09:10.973075 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2926f79c_8d6d_4013_9953_bfe9d962d618.slice/crio-3dd42d5a03f15c2e7d57efb4da1833c746cdd7cce238ba81bdc492bb2ef77e1b WatchSource:0}: Error finding container 3dd42d5a03f15c2e7d57efb4da1833c746cdd7cce238ba81bdc492bb2ef77e1b: Status 404 returned error can't find the container with id 3dd42d5a03f15c2e7d57efb4da1833c746cdd7cce238ba81bdc492bb2ef77e1b Jul 27 17:09:11.169004 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:11.168917 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" event={"ID":"2926f79c-8d6d-4013-9953-bfe9d962d618","Type":"ContainerStarted","Data":"3dd42d5a03f15c2e7d57efb4da1833c746cdd7cce238ba81bdc492bb2ef77e1b"} Jul 27 17:09:12.175933 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:12.175902 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" event={"ID":"2926f79c-8d6d-4013-9953-bfe9d962d618","Type":"ContainerStarted","Data":"7dea8e9473b3437829a5dfb81d4722db92db3226a08c124ca6230618e740b672"} Jul 27 17:09:12.196417 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:12.196334 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" podStartSLOduration=1.955880686 podStartE2EDuration="2.196316757s" totalImagesPullingTime="240.436071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:09:12.195397373 +0000 UTC m=+758.479556656" watchObservedRunningTime="2026-07-27 17:09:12.196316757 +0000 UTC m=+758.480476043" Jul 27 17:09:20.836676 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:20.836635 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:20.837168 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:20.836714 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:20.848941 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:20.848909 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:09:21.237173 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:09:21.237092 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-47tk5" Jul 27 17:10:57.194460 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.194430 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6458d6b58d-chc22"] Jul 27 17:10:57.197909 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.197891 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.202566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.202539 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t2r6d\"" Jul 27 17:10:57.202566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.202560 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 27 17:10:57.221589 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.221555 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6458d6b58d-chc22"] Jul 27 17:10:57.240525 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.240493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cb053cd-d164-46d4-ad76-ab0935841b0e-tls-cert\") pod \"authorino-6458d6b58d-chc22\" (UID: \"3cb053cd-d164-46d4-ad76-ab0935841b0e\") " pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.240681 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.240549 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr59v\" (UniqueName: \"kubernetes.io/projected/3cb053cd-d164-46d4-ad76-ab0935841b0e-kube-api-access-vr59v\") pod \"authorino-6458d6b58d-chc22\" (UID: \"3cb053cd-d164-46d4-ad76-ab0935841b0e\") " pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.342104 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.342071 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr59v\" (UniqueName: \"kubernetes.io/projected/3cb053cd-d164-46d4-ad76-ab0935841b0e-kube-api-access-vr59v\") pod \"authorino-6458d6b58d-chc22\" (UID: \"3cb053cd-d164-46d4-ad76-ab0935841b0e\") " pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.342261 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.342170 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cb053cd-d164-46d4-ad76-ab0935841b0e-tls-cert\") pod \"authorino-6458d6b58d-chc22\" (UID: \"3cb053cd-d164-46d4-ad76-ab0935841b0e\") " pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.344513 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.344494 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3cb053cd-d164-46d4-ad76-ab0935841b0e-tls-cert\") pod \"authorino-6458d6b58d-chc22\" (UID: \"3cb053cd-d164-46d4-ad76-ab0935841b0e\") " pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.350848 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.350824 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr59v\" (UniqueName: \"kubernetes.io/projected/3cb053cd-d164-46d4-ad76-ab0935841b0e-kube-api-access-vr59v\") pod \"authorino-6458d6b58d-chc22\" (UID: \"3cb053cd-d164-46d4-ad76-ab0935841b0e\") " pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.507446 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.507347 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6458d6b58d-chc22" Jul 27 17:10:57.664060 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.664024 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6458d6b58d-chc22"] Jul 27 17:10:57.665187 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:10:57.665164 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cb053cd_d164_46d4_ad76_ab0935841b0e.slice/crio-7ebd9db1d4d529154213b6a59f5eb87feea595f4c60b6fb5fff0ab293a4f4ea3 WatchSource:0}: Error finding container 7ebd9db1d4d529154213b6a59f5eb87feea595f4c60b6fb5fff0ab293a4f4ea3: Status 404 returned error can't find the container with id 7ebd9db1d4d529154213b6a59f5eb87feea595f4c60b6fb5fff0ab293a4f4ea3 Jul 27 17:10:57.773933 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:57.773848 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6458d6b58d-chc22" event={"ID":"3cb053cd-d164-46d4-ad76-ab0935841b0e","Type":"ContainerStarted","Data":"7ebd9db1d4d529154213b6a59f5eb87feea595f4c60b6fb5fff0ab293a4f4ea3"} Jul 27 17:10:58.780907 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:58.780801 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6458d6b58d-chc22" event={"ID":"3cb053cd-d164-46d4-ad76-ab0935841b0e","Type":"ContainerStarted","Data":"df196cdfe1d4b57c28380dd17c15fcf9f580eaaa60263dda9556393f0add2c14"} Jul 27 17:10:58.812452 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:10:58.812396 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6458d6b58d-chc22" podStartSLOduration=0.959497575 podStartE2EDuration="1.812363109s" totalImagesPullingTime="852.865534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:10:58.811090227 +0000 UTC m=+865.095249500" watchObservedRunningTime="2026-07-27 17:10:58.812363109 +0000 UTC m=+865.096522392" Jul 27 17:27:02.529753 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.529718 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt"] Jul 27 17:27:02.533494 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.533472 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:02.536723 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.536696 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:27:02.541471 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.541447 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt"] Jul 27 17:27:02.666482 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.666447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt\" (UID: \"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:02.767255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.767219 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt\" (UID: \"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:02.767845 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.767823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt\" (UID: \"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:02.846022 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.845976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:02.988336 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.988306 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt"] Jul 27 17:27:02.989830 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:27:02.989805 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb3a9a7_600c_4a1b_a524_fa1d2aa9a376.slice/crio-4d729b47d5bf50e15ff329fe9db19ef2234c7e0bed5bc29d95da0a46ff701b50 WatchSource:0}: Error finding container 4d729b47d5bf50e15ff329fe9db19ef2234c7e0bed5bc29d95da0a46ff701b50: Status 404 returned error can't find the container with id 4d729b47d5bf50e15ff329fe9db19ef2234c7e0bed5bc29d95da0a46ff701b50 Jul 27 17:27:02.991715 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:02.991696 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 17:27:03.391060 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:03.391026 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" event={"ID":"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376","Type":"ContainerStarted","Data":"4d729b47d5bf50e15ff329fe9db19ef2234c7e0bed5bc29d95da0a46ff701b50"} Jul 27 17:27:07.416747 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:07.416713 2582 generic.go:356] "Generic (PLEG): container finished" podID="bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376" containerID="4e97876f7b4d1a1a92a97c3feabb12e055093097758a2118d81c4b207b82b33b" exitCode=0 Jul 27 17:27:07.417127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:07.416800 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" event={"ID":"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376","Type":"ContainerDied","Data":"4e97876f7b4d1a1a92a97c3feabb12e055093097758a2118d81c4b207b82b33b"} Jul 27 17:27:08.560492 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:08.560461 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:08.728774 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:08.728682 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376-maas-api-service-ca\") pod \"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376\" (UID: \"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376\") " Jul 27 17:27:08.729074 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:08.729047 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376" (UID: "bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:27:08.830341 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:08.830299 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:27:09.429562 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:09.429532 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" Jul 27 17:27:09.429772 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:09.429557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt" event={"ID":"bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376","Type":"ContainerDied","Data":"4d729b47d5bf50e15ff329fe9db19ef2234c7e0bed5bc29d95da0a46ff701b50"} Jul 27 17:27:09.429772 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:09.429607 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d729b47d5bf50e15ff329fe9db19ef2234c7e0bed5bc29d95da0a46ff701b50" Jul 27 17:27:17.648345 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:17.648304 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt"] Jul 27 17:27:17.653203 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:17.653175 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41d9ef37-qrcvt"] Jul 27 17:27:18.198822 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:18.198785 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376" path="/var/lib/kubelet/pods/bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376/volumes" Jul 27 17:27:46.355529 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.355490 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt"] Jul 27 17:27:46.355968 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.355900 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376" containerName="revoke-keys" Jul 27 17:27:46.355968 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.355912 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376" containerName="revoke-keys" Jul 27 17:27:46.356046 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.356006 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdb3a9a7-600c-4a1b-a524-fa1d2aa9a376" containerName="revoke-keys" Jul 27 17:27:46.363748 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.363725 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:46.368954 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.368935 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:27:46.373319 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.373281 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt"] Jul 27 17:27:46.499713 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.499670 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0611b7d4-1a8f-44b5-84a7-13b5857865a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26960311-gx7nt\" (UID: \"0611b7d4-1a8f-44b5-84a7-13b5857865a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:46.601101 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.601064 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0611b7d4-1a8f-44b5-84a7-13b5857865a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26960311-gx7nt\" (UID: \"0611b7d4-1a8f-44b5-84a7-13b5857865a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:46.601749 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.601727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0611b7d4-1a8f-44b5-84a7-13b5857865a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26960311-gx7nt\" (UID: \"0611b7d4-1a8f-44b5-84a7-13b5857865a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:46.676536 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.676448 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:46.814949 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:46.814911 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt"] Jul 27 17:27:46.816154 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:27:46.816125 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0611b7d4_1a8f_44b5_84a7_13b5857865a9.slice/crio-729ed2e2ece6f55ae8612d42715f96926e96332325a7779efb9de76a54d1a114 WatchSource:0}: Error finding container 729ed2e2ece6f55ae8612d42715f96926e96332325a7779efb9de76a54d1a114: Status 404 returned error can't find the container with id 729ed2e2ece6f55ae8612d42715f96926e96332325a7779efb9de76a54d1a114 Jul 27 17:27:47.645266 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:47.645230 2582 generic.go:356] "Generic (PLEG): container finished" podID="0611b7d4-1a8f-44b5-84a7-13b5857865a9" containerID="d1a47c12d205cd8b0934b2e751980632641d8ab2125fc04943c0cfd3cc7f2411" exitCode=0 Jul 27 17:27:47.645266 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:47.645267 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" event={"ID":"0611b7d4-1a8f-44b5-84a7-13b5857865a9","Type":"ContainerDied","Data":"d1a47c12d205cd8b0934b2e751980632641d8ab2125fc04943c0cfd3cc7f2411"} Jul 27 17:27:47.645728 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:47.645304 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" event={"ID":"0611b7d4-1a8f-44b5-84a7-13b5857865a9","Type":"ContainerStarted","Data":"729ed2e2ece6f55ae8612d42715f96926e96332325a7779efb9de76a54d1a114"} Jul 27 17:27:48.787653 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:48.787627 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:48.922597 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:48.922501 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0611b7d4-1a8f-44b5-84a7-13b5857865a9-maas-api-service-ca\") pod \"0611b7d4-1a8f-44b5-84a7-13b5857865a9\" (UID: \"0611b7d4-1a8f-44b5-84a7-13b5857865a9\") " Jul 27 17:27:48.922878 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:48.922855 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0611b7d4-1a8f-44b5-84a7-13b5857865a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0611b7d4-1a8f-44b5-84a7-13b5857865a9" (UID: "0611b7d4-1a8f-44b5-84a7-13b5857865a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:27:49.023910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:49.023868 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0611b7d4-1a8f-44b5-84a7-13b5857865a9-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:27:49.658308 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:49.658274 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" Jul 27 17:27:49.658308 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:49.658298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt" event={"ID":"0611b7d4-1a8f-44b5-84a7-13b5857865a9","Type":"ContainerDied","Data":"729ed2e2ece6f55ae8612d42715f96926e96332325a7779efb9de76a54d1a114"} Jul 27 17:27:49.658581 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:27:49.658372 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="729ed2e2ece6f55ae8612d42715f96926e96332325a7779efb9de76a54d1a114" Jul 27 17:28:01.545467 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:01.545356 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt"] Jul 27 17:28:01.554405 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:01.548931 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26960311-gx7nt"] Jul 27 17:28:02.198926 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:02.198891 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0611b7d4-1a8f-44b5-84a7-13b5857865a9" path="/var/lib/kubelet/pods/0611b7d4-1a8f-44b5-84a7-13b5857865a9/volumes" Jul 27 17:28:30.882716 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.882680 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l"] Jul 27 17:28:30.883182 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.883093 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0611b7d4-1a8f-44b5-84a7-13b5857865a9" containerName="revoke-keys" Jul 27 17:28:30.883182 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.883103 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0611b7d4-1a8f-44b5-84a7-13b5857865a9" containerName="revoke-keys" Jul 27 17:28:30.883259 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.883185 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0611b7d4-1a8f-44b5-84a7-13b5857865a9" containerName="revoke-keys" Jul 27 17:28:30.886538 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.886519 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:30.890059 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.890036 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:28:30.895264 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.895235 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l"] Jul 27 17:28:30.930734 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:30.930706 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f430d66b-724f-4b71-9d2f-3e9001068bfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l\" (UID: \"f430d66b-724f-4b71-9d2f-3e9001068bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:31.031793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.031746 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f430d66b-724f-4b71-9d2f-3e9001068bfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l\" (UID: \"f430d66b-724f-4b71-9d2f-3e9001068bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:31.032337 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.032314 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f430d66b-724f-4b71-9d2f-3e9001068bfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l\" (UID: \"f430d66b-724f-4b71-9d2f-3e9001068bfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:31.198558 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.198466 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:31.337715 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.337677 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l"] Jul 27 17:28:31.338839 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:28:31.338813 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf430d66b_724f_4b71_9d2f_3e9001068bfc.slice/crio-a124c459cc0788c017f99d2b52172207b11e8e18aff723773274dad9a281bc8d WatchSource:0}: Error finding container a124c459cc0788c017f99d2b52172207b11e8e18aff723773274dad9a281bc8d: Status 404 returned error can't find the container with id a124c459cc0788c017f99d2b52172207b11e8e18aff723773274dad9a281bc8d Jul 27 17:28:31.903576 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.903533 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" event={"ID":"f430d66b-724f-4b71-9d2f-3e9001068bfc","Type":"ContainerStarted","Data":"70fdbe76964ae3ecbece3dd2fbb39af6c8f6a916a36fb026f8fd8dab693b1f52"} Jul 27 17:28:31.903949 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.903600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" event={"ID":"f430d66b-724f-4b71-9d2f-3e9001068bfc","Type":"ContainerStarted","Data":"a124c459cc0788c017f99d2b52172207b11e8e18aff723773274dad9a281bc8d"} Jul 27 17:28:31.922284 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:31.922233 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" podStartSLOduration=1.922217399 podStartE2EDuration="1.922217399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:28:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:28:31.921316155 +0000 UTC m=+1918.205475429" watchObservedRunningTime="2026-07-27 17:28:31.922217399 +0000 UTC m=+1918.206376681" Jul 27 17:28:32.910625 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:32.910593 2582 generic.go:356] "Generic (PLEG): container finished" podID="f430d66b-724f-4b71-9d2f-3e9001068bfc" containerID="70fdbe76964ae3ecbece3dd2fbb39af6c8f6a916a36fb026f8fd8dab693b1f52" exitCode=0 Jul 27 17:28:32.911076 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:32.910680 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" event={"ID":"f430d66b-724f-4b71-9d2f-3e9001068bfc","Type":"ContainerDied","Data":"70fdbe76964ae3ecbece3dd2fbb39af6c8f6a916a36fb026f8fd8dab693b1f52"} Jul 27 17:28:34.051528 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.051498 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:34.156759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.156720 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f430d66b-724f-4b71-9d2f-3e9001068bfc-maas-api-service-ca\") pod \"f430d66b-724f-4b71-9d2f-3e9001068bfc\" (UID: \"f430d66b-724f-4b71-9d2f-3e9001068bfc\") " Jul 27 17:28:34.157070 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.157046 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f430d66b-724f-4b71-9d2f-3e9001068bfc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f430d66b-724f-4b71-9d2f-3e9001068bfc" (UID: "f430d66b-724f-4b71-9d2f-3e9001068bfc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:28:34.257464 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.257436 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f430d66b-724f-4b71-9d2f-3e9001068bfc-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:28:34.923605 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.923570 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" Jul 27 17:28:34.923782 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.923575 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l" event={"ID":"f430d66b-724f-4b71-9d2f-3e9001068bfc","Type":"ContainerDied","Data":"a124c459cc0788c017f99d2b52172207b11e8e18aff723773274dad9a281bc8d"} Jul 27 17:28:34.923782 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:34.923697 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a124c459cc0788c017f99d2b52172207b11e8e18aff723773274dad9a281bc8d" Jul 27 17:28:45.981745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:45.981713 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l"] Jul 27 17:28:45.984122 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:45.984095 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eda34c7f-zrk9l"] Jul 27 17:28:46.198824 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:28:46.198788 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f430d66b-724f-4b71-9d2f-3e9001068bfc" path="/var/lib/kubelet/pods/f430d66b-724f-4b71-9d2f-3e9001068bfc/volumes" Jul 27 17:29:15.567609 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.564459 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b"] Jul 27 17:29:15.567609 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.565417 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f430d66b-724f-4b71-9d2f-3e9001068bfc" containerName="revoke-keys" Jul 27 17:29:15.567609 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.565441 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f430d66b-724f-4b71-9d2f-3e9001068bfc" containerName="revoke-keys" Jul 27 17:29:15.567609 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.565626 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f430d66b-724f-4b71-9d2f-3e9001068bfc" containerName="revoke-keys" Jul 27 17:29:15.569636 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.569610 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:15.573071 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.573046 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:29:15.574635 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.574607 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b"] Jul 27 17:29:15.755534 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.755492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-834e32af-s784b\" (UID: \"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:15.856855 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.856817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-834e32af-s784b\" (UID: \"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:15.857578 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.857554 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-834e32af-s784b\" (UID: \"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:15.882218 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:15.882186 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:16.021196 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:16.021169 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b"] Jul 27 17:29:16.023026 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:29:16.022994 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0b3f57_7979_4d5c_89d9_a87c49a37ea3.slice/crio-73fc8485439da2596aaca513d5f1d343a21e803581a56d65c21de05a619efcb7 WatchSource:0}: Error finding container 73fc8485439da2596aaca513d5f1d343a21e803581a56d65c21de05a619efcb7: Status 404 returned error can't find the container with id 73fc8485439da2596aaca513d5f1d343a21e803581a56d65c21de05a619efcb7 Jul 27 17:29:16.162576 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:16.162477 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" event={"ID":"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3","Type":"ContainerStarted","Data":"73fc8485439da2596aaca513d5f1d343a21e803581a56d65c21de05a619efcb7"} Jul 27 17:29:17.170005 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:17.169968 2582 generic.go:356] "Generic (PLEG): container finished" podID="bb0b3f57-7979-4d5c-89d9-a87c49a37ea3" containerID="9db867dd1990a3de2229d298f04f504bd74bc93dd40cc87696c9e06af45fef04" exitCode=0 Jul 27 17:29:17.170462 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:17.170054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" event={"ID":"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3","Type":"ContainerDied","Data":"9db867dd1990a3de2229d298f04f504bd74bc93dd40cc87696c9e06af45fef04"} Jul 27 17:29:18.309491 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:18.309467 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:18.377833 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:18.377803 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3-maas-api-service-ca\") pod \"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3\" (UID: \"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3\") " Jul 27 17:29:18.378157 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:18.378133 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb0b3f57-7979-4d5c-89d9-a87c49a37ea3" (UID: "bb0b3f57-7979-4d5c-89d9-a87c49a37ea3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:29:18.479542 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:18.479459 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:29:19.183436 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:19.183403 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" event={"ID":"bb0b3f57-7979-4d5c-89d9-a87c49a37ea3","Type":"ContainerDied","Data":"73fc8485439da2596aaca513d5f1d343a21e803581a56d65c21de05a619efcb7"} Jul 27 17:29:19.183436 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:19.183427 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b" Jul 27 17:29:19.183640 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:19.183450 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73fc8485439da2596aaca513d5f1d343a21e803581a56d65c21de05a619efcb7" Jul 27 17:29:30.659237 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:30.659204 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b"] Jul 27 17:29:30.661503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:30.661480 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-834e32af-s784b"] Jul 27 17:29:32.200276 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:29:32.200242 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb0b3f57-7979-4d5c-89d9-a87c49a37ea3" path="/var/lib/kubelet/pods/bb0b3f57-7979-4d5c-89d9-a87c49a37ea3/volumes" Jul 27 17:31:26.282503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.282417 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr"] Jul 27 17:31:26.283093 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.283046 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb0b3f57-7979-4d5c-89d9-a87c49a37ea3" containerName="revoke-keys" Jul 27 17:31:26.283093 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.283070 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb0b3f57-7979-4d5c-89d9-a87c49a37ea3" containerName="revoke-keys" Jul 27 17:31:26.283214 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.283187 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb0b3f57-7979-4d5c-89d9-a87c49a37ea3" containerName="revoke-keys" Jul 27 17:31:26.286420 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.286400 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:26.290449 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.290433 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:31:26.296612 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.296585 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr"] Jul 27 17:31:26.317755 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.317723 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad30fa7-a1b9-4054-af01-e9f7285a00b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18117344-76hvr\" (UID: \"6ad30fa7-a1b9-4054-af01-e9f7285a00b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:26.419206 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.419160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad30fa7-a1b9-4054-af01-e9f7285a00b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18117344-76hvr\" (UID: \"6ad30fa7-a1b9-4054-af01-e9f7285a00b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:26.419846 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.419824 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad30fa7-a1b9-4054-af01-e9f7285a00b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-18117344-76hvr\" (UID: \"6ad30fa7-a1b9-4054-af01-e9f7285a00b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:26.598117 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.598078 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:26.736837 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.736807 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr"] Jul 27 17:31:26.738453 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:31:26.738427 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad30fa7_a1b9_4054_af01_e9f7285a00b7.slice/crio-17264ea485782db80c69cc90a4caace0110f503e98d667f18f5d04d4dd192ad9 WatchSource:0}: Error finding container 17264ea485782db80c69cc90a4caace0110f503e98d667f18f5d04d4dd192ad9: Status 404 returned error can't find the container with id 17264ea485782db80c69cc90a4caace0110f503e98d667f18f5d04d4dd192ad9 Jul 27 17:31:26.953718 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:26.953681 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" event={"ID":"6ad30fa7-a1b9-4054-af01-e9f7285a00b7","Type":"ContainerStarted","Data":"17264ea485782db80c69cc90a4caace0110f503e98d667f18f5d04d4dd192ad9"} Jul 27 17:31:27.961473 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:27.961433 2582 generic.go:356] "Generic (PLEG): container finished" podID="6ad30fa7-a1b9-4054-af01-e9f7285a00b7" containerID="65fd21a13e3c25650a9482d3f81d2ad361ff8b69dac7f5878c92f22d582b74ed" exitCode=0 Jul 27 17:31:27.961881 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:27.961527 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" event={"ID":"6ad30fa7-a1b9-4054-af01-e9f7285a00b7","Type":"ContainerDied","Data":"65fd21a13e3c25650a9482d3f81d2ad361ff8b69dac7f5878c92f22d582b74ed"} Jul 27 17:31:29.099117 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.099093 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:29.145947 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.145915 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad30fa7-a1b9-4054-af01-e9f7285a00b7-maas-api-service-ca\") pod \"6ad30fa7-a1b9-4054-af01-e9f7285a00b7\" (UID: \"6ad30fa7-a1b9-4054-af01-e9f7285a00b7\") " Jul 27 17:31:29.146273 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.146251 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad30fa7-a1b9-4054-af01-e9f7285a00b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ad30fa7-a1b9-4054-af01-e9f7285a00b7" (UID: "6ad30fa7-a1b9-4054-af01-e9f7285a00b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:31:29.247453 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.247347 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ad30fa7-a1b9-4054-af01-e9f7285a00b7-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:31:29.975412 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.975356 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" event={"ID":"6ad30fa7-a1b9-4054-af01-e9f7285a00b7","Type":"ContainerDied","Data":"17264ea485782db80c69cc90a4caace0110f503e98d667f18f5d04d4dd192ad9"} Jul 27 17:31:29.975412 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.975413 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr" Jul 27 17:31:29.975626 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:29.975433 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17264ea485782db80c69cc90a4caace0110f503e98d667f18f5d04d4dd192ad9" Jul 27 17:31:41.381443 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:41.381405 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr"] Jul 27 17:31:41.383773 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:41.383751 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-18117344-76hvr"] Jul 27 17:31:42.200092 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:31:42.200059 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad30fa7-a1b9-4054-af01-e9f7285a00b7" path="/var/lib/kubelet/pods/6ad30fa7-a1b9-4054-af01-e9f7285a00b7/volumes" Jul 27 17:33:01.241277 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.241243 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf"] Jul 27 17:33:01.241729 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.241697 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad30fa7-a1b9-4054-af01-e9f7285a00b7" containerName="revoke-keys" Jul 27 17:33:01.241729 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.241710 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad30fa7-a1b9-4054-af01-e9f7285a00b7" containerName="revoke-keys" Jul 27 17:33:01.241818 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.241803 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad30fa7-a1b9-4054-af01-e9f7285a00b7" containerName="revoke-keys" Jul 27 17:33:01.244949 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.244933 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:01.248339 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.248320 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:33:01.254350 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.254321 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf"] Jul 27 17:33:01.329146 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.329087 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbaf8615-1a87-46b5-9657-c0f9bd868877-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-736d5a21-j24pf\" (UID: \"dbaf8615-1a87-46b5-9657-c0f9bd868877\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:01.430446 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.430409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbaf8615-1a87-46b5-9657-c0f9bd868877-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-736d5a21-j24pf\" (UID: \"dbaf8615-1a87-46b5-9657-c0f9bd868877\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:01.431072 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.431052 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbaf8615-1a87-46b5-9657-c0f9bd868877-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-736d5a21-j24pf\" (UID: \"dbaf8615-1a87-46b5-9657-c0f9bd868877\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:01.557128 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.557042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:01.696723 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:01.696697 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf"] Jul 27 17:33:01.698656 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:33:01.698629 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbaf8615_1a87_46b5_9657_c0f9bd868877.slice/crio-0dac9fc3ae07493739cfedc8c2658867ff335c631aff656f777ad9b1d068a8f7 WatchSource:0}: Error finding container 0dac9fc3ae07493739cfedc8c2658867ff335c631aff656f777ad9b1d068a8f7: Status 404 returned error can't find the container with id 0dac9fc3ae07493739cfedc8c2658867ff335c631aff656f777ad9b1d068a8f7 Jul 27 17:33:02.511228 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:02.511189 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" event={"ID":"dbaf8615-1a87-46b5-9657-c0f9bd868877","Type":"ContainerStarted","Data":"65a45ac66c6598935ec4d6cfad54147b32bc5c9a456b5c89eed935ece52c53d3"} Jul 27 17:33:02.511581 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:02.511261 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" event={"ID":"dbaf8615-1a87-46b5-9657-c0f9bd868877","Type":"ContainerStarted","Data":"0dac9fc3ae07493739cfedc8c2658867ff335c631aff656f777ad9b1d068a8f7"} Jul 27 17:33:02.531208 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:02.531158 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" podStartSLOduration=1.5311409120000001 podStartE2EDuration="1.531140912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:33:02.52818298 +0000 UTC m=+2188.812342263" watchObservedRunningTime="2026-07-27 17:33:02.531140912 +0000 UTC m=+2188.815300194" Jul 27 17:33:03.519295 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:03.519258 2582 generic.go:356] "Generic (PLEG): container finished" podID="dbaf8615-1a87-46b5-9657-c0f9bd868877" containerID="65a45ac66c6598935ec4d6cfad54147b32bc5c9a456b5c89eed935ece52c53d3" exitCode=0 Jul 27 17:33:03.519731 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:03.519342 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" event={"ID":"dbaf8615-1a87-46b5-9657-c0f9bd868877","Type":"ContainerDied","Data":"65a45ac66c6598935ec4d6cfad54147b32bc5c9a456b5c89eed935ece52c53d3"} Jul 27 17:33:04.664209 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:04.664182 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:04.763105 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:04.763070 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbaf8615-1a87-46b5-9657-c0f9bd868877-maas-api-service-ca\") pod \"dbaf8615-1a87-46b5-9657-c0f9bd868877\" (UID: \"dbaf8615-1a87-46b5-9657-c0f9bd868877\") " Jul 27 17:33:04.763438 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:04.763415 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbaf8615-1a87-46b5-9657-c0f9bd868877-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbaf8615-1a87-46b5-9657-c0f9bd868877" (UID: "dbaf8615-1a87-46b5-9657-c0f9bd868877"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:33:04.763557 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:04.763527 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbaf8615-1a87-46b5-9657-c0f9bd868877-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:33:05.532882 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:05.532847 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" Jul 27 17:33:05.532882 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:05.532863 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf" event={"ID":"dbaf8615-1a87-46b5-9657-c0f9bd868877","Type":"ContainerDied","Data":"0dac9fc3ae07493739cfedc8c2658867ff335c631aff656f777ad9b1d068a8f7"} Jul 27 17:33:05.533102 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:05.532915 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dac9fc3ae07493739cfedc8c2658867ff335c631aff656f777ad9b1d068a8f7" Jul 27 17:33:21.341137 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:21.341104 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf"] Jul 27 17:33:21.344665 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:21.344637 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-736d5a21-j24pf"] Jul 27 17:33:22.198643 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:22.198606 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbaf8615-1a87-46b5-9657-c0f9bd868877" path="/var/lib/kubelet/pods/dbaf8615-1a87-46b5-9657-c0f9bd868877/volumes" Jul 27 17:33:40.355977 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:33:40.355945 2582 scope.go:117] "RemoveContainer" containerID="4e97876f7b4d1a1a92a97c3feabb12e055093097758a2118d81c4b207b82b33b" Jul 27 17:34:40.378471 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:34:40.378441 2582 scope.go:117] "RemoveContainer" containerID="70fdbe76964ae3ecbece3dd2fbb39af6c8f6a916a36fb026f8fd8dab693b1f52" Jul 27 17:34:40.387226 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:34:40.387190 2582 scope.go:117] "RemoveContainer" containerID="d1a47c12d205cd8b0934b2e751980632641d8ab2125fc04943c0cfd3cc7f2411" Jul 27 17:35:40.415227 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:35:40.415192 2582 scope.go:117] "RemoveContainer" containerID="9db867dd1990a3de2229d298f04f504bd74bc93dd40cc87696c9e06af45fef04" Jul 27 17:36:37.623667 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.623629 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw"] Jul 27 17:36:37.624273 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.624257 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbaf8615-1a87-46b5-9657-c0f9bd868877" containerName="revoke-keys" Jul 27 17:36:37.624355 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.624277 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbaf8615-1a87-46b5-9657-c0f9bd868877" containerName="revoke-keys" Jul 27 17:36:37.624466 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.624398 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbaf8615-1a87-46b5-9657-c0f9bd868877" containerName="revoke-keys" Jul 27 17:36:37.627616 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.627594 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:37.630682 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.630656 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:36:37.634847 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.634780 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw"] Jul 27 17:36:37.732194 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.732152 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/653fb0a8-ca35-4e39-ba2a-69acb82318a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw\" (UID: \"653fb0a8-ca35-4e39-ba2a-69acb82318a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:37.833732 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.833691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/653fb0a8-ca35-4e39-ba2a-69acb82318a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw\" (UID: \"653fb0a8-ca35-4e39-ba2a-69acb82318a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:37.834321 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.834301 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/653fb0a8-ca35-4e39-ba2a-69acb82318a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw\" (UID: \"653fb0a8-ca35-4e39-ba2a-69acb82318a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:37.940910 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:37.940805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:38.079511 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:38.079481 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw"] Jul 27 17:36:38.081259 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:36:38.081237 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653fb0a8_ca35_4e39_ba2a_69acb82318a1.slice/crio-d5da43d4bb06ca9c3a341c36d59a49f686b242204754279ed3c70d769d6d1c63 WatchSource:0}: Error finding container d5da43d4bb06ca9c3a341c36d59a49f686b242204754279ed3c70d769d6d1c63: Status 404 returned error can't find the container with id d5da43d4bb06ca9c3a341c36d59a49f686b242204754279ed3c70d769d6d1c63 Jul 27 17:36:38.775243 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:38.775205 2582 generic.go:356] "Generic (PLEG): container finished" podID="653fb0a8-ca35-4e39-ba2a-69acb82318a1" containerID="aadedd9bb8d31c37e8b264d9fe80f57c4f25583093097351bbb7884b30cec18b" exitCode=0 Jul 27 17:36:38.775675 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:38.775300 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" event={"ID":"653fb0a8-ca35-4e39-ba2a-69acb82318a1","Type":"ContainerDied","Data":"aadedd9bb8d31c37e8b264d9fe80f57c4f25583093097351bbb7884b30cec18b"} Jul 27 17:36:38.775675 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:38.775419 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" event={"ID":"653fb0a8-ca35-4e39-ba2a-69acb82318a1","Type":"ContainerStarted","Data":"d5da43d4bb06ca9c3a341c36d59a49f686b242204754279ed3c70d769d6d1c63"} Jul 27 17:36:39.923583 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:39.923555 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:40.055454 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:40.055320 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/653fb0a8-ca35-4e39-ba2a-69acb82318a1-maas-api-service-ca\") pod \"653fb0a8-ca35-4e39-ba2a-69acb82318a1\" (UID: \"653fb0a8-ca35-4e39-ba2a-69acb82318a1\") " Jul 27 17:36:40.055698 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:40.055674 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/653fb0a8-ca35-4e39-ba2a-69acb82318a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "653fb0a8-ca35-4e39-ba2a-69acb82318a1" (UID: "653fb0a8-ca35-4e39-ba2a-69acb82318a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:36:40.156612 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:40.156576 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/653fb0a8-ca35-4e39-ba2a-69acb82318a1-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:36:40.788355 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:40.788327 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" Jul 27 17:36:40.788534 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:40.788369 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw" event={"ID":"653fb0a8-ca35-4e39-ba2a-69acb82318a1","Type":"ContainerDied","Data":"d5da43d4bb06ca9c3a341c36d59a49f686b242204754279ed3c70d769d6d1c63"} Jul 27 17:36:40.788534 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:40.788439 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5da43d4bb06ca9c3a341c36d59a49f686b242204754279ed3c70d769d6d1c63" Jul 27 17:36:52.711954 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:52.711923 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw"] Jul 27 17:36:52.714387 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:52.714352 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-a6f598-d4rzw"] Jul 27 17:36:54.200331 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:54.200297 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="653fb0a8-ca35-4e39-ba2a-69acb82318a1" path="/var/lib/kubelet/pods/653fb0a8-ca35-4e39-ba2a-69acb82318a1/volumes" Jul 27 17:36:59.769627 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.769541 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w"] Jul 27 17:36:59.770054 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.769977 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="653fb0a8-ca35-4e39-ba2a-69acb82318a1" containerName="revoke-keys" Jul 27 17:36:59.770054 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.769987 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="653fb0a8-ca35-4e39-ba2a-69acb82318a1" containerName="revoke-keys" Jul 27 17:36:59.770127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.770056 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="653fb0a8-ca35-4e39-ba2a-69acb82318a1" containerName="revoke-keys" Jul 27 17:36:59.773477 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.773460 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:36:59.776729 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.776710 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:36:59.784566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.784539 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w"] Jul 27 17:36:59.840640 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.840607 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w\" (UID: \"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:36:59.941510 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.941473 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w\" (UID: \"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:36:59.942239 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:36:59.942216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w\" (UID: \"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:37:00.084886 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:00.084854 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:37:00.229334 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:00.229299 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w"] Jul 27 17:37:00.231341 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:37:00.231314 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134ee4b3_83bb_49b8_8ee2_b91e34d84cdd.slice/crio-aa1761f6ebbeafaa2875f52087e4a62f992e9ead40230f6f1a09403303de32ca WatchSource:0}: Error finding container aa1761f6ebbeafaa2875f52087e4a62f992e9ead40230f6f1a09403303de32ca: Status 404 returned error can't find the container with id aa1761f6ebbeafaa2875f52087e4a62f992e9ead40230f6f1a09403303de32ca Jul 27 17:37:00.904127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:00.904092 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" event={"ID":"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd","Type":"ContainerStarted","Data":"f0536ddc8364d9e7b5d286b1cd401100023d2907f4fcbe894b3fe8ee6690c8a0"} Jul 27 17:37:00.904529 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:00.904152 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" event={"ID":"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd","Type":"ContainerStarted","Data":"aa1761f6ebbeafaa2875f52087e4a62f992e9ead40230f6f1a09403303de32ca"} Jul 27 17:37:00.922709 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:00.922655 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" podStartSLOduration=1.922638812 podStartE2EDuration="1.922638812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:37:00.920279564 +0000 UTC m=+2427.204438847" watchObservedRunningTime="2026-07-27 17:37:00.922638812 +0000 UTC m=+2427.206798095" Jul 27 17:37:01.911640 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:01.911604 2582 generic.go:356] "Generic (PLEG): container finished" podID="134ee4b3-83bb-49b8-8ee2-b91e34d84cdd" containerID="f0536ddc8364d9e7b5d286b1cd401100023d2907f4fcbe894b3fe8ee6690c8a0" exitCode=0 Jul 27 17:37:01.912030 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:01.911700 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" event={"ID":"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd","Type":"ContainerDied","Data":"f0536ddc8364d9e7b5d286b1cd401100023d2907f4fcbe894b3fe8ee6690c8a0"} Jul 27 17:37:03.058902 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.058874 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:37:03.172025 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.171927 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd-maas-api-service-ca\") pod \"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd\" (UID: \"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd\") " Jul 27 17:37:03.172300 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.172274 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "134ee4b3-83bb-49b8-8ee2-b91e34d84cdd" (UID: "134ee4b3-83bb-49b8-8ee2-b91e34d84cdd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:37:03.273150 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.273082 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:37:03.925576 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.925545 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" event={"ID":"134ee4b3-83bb-49b8-8ee2-b91e34d84cdd","Type":"ContainerDied","Data":"aa1761f6ebbeafaa2875f52087e4a62f992e9ead40230f6f1a09403303de32ca"} Jul 27 17:37:03.925761 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.925581 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w" Jul 27 17:37:03.925761 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:03.925594 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa1761f6ebbeafaa2875f52087e4a62f992e9ead40230f6f1a09403303de32ca" Jul 27 17:37:14.848303 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:14.848263 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w"] Jul 27 17:37:14.853282 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:14.853240 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7a55cb-gmt5w"] Jul 27 17:37:16.199549 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:16.199513 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="134ee4b3-83bb-49b8-8ee2-b91e34d84cdd" path="/var/lib/kubelet/pods/134ee4b3-83bb-49b8-8ee2-b91e34d84cdd/volumes" Jul 27 17:37:40.446661 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:40.446623 2582 scope.go:117] "RemoveContainer" containerID="65fd21a13e3c25650a9482d3f81d2ad361ff8b69dac7f5878c92f22d582b74ed" Jul 27 17:37:46.048800 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.048764 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj"] Jul 27 17:37:46.049267 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.049160 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134ee4b3-83bb-49b8-8ee2-b91e34d84cdd" containerName="revoke-keys" Jul 27 17:37:46.049267 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.049172 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="134ee4b3-83bb-49b8-8ee2-b91e34d84cdd" containerName="revoke-keys" Jul 27 17:37:46.049267 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.049242 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="134ee4b3-83bb-49b8-8ee2-b91e34d84cdd" containerName="revoke-keys" Jul 27 17:37:46.052405 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.052368 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:46.055370 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.055350 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:37:46.061238 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.061215 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj"] Jul 27 17:37:46.207135 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.207102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f9d7c6-94d4-4607-a5a6-1dbfa717f907-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj\" (UID: \"41f9d7c6-94d4-4607-a5a6-1dbfa717f907\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:46.308021 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.307938 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f9d7c6-94d4-4607-a5a6-1dbfa717f907-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj\" (UID: \"41f9d7c6-94d4-4607-a5a6-1dbfa717f907\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:46.308600 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.308577 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f9d7c6-94d4-4607-a5a6-1dbfa717f907-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj\" (UID: \"41f9d7c6-94d4-4607-a5a6-1dbfa717f907\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:46.364952 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.364917 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:46.509660 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:46.509633 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj"] Jul 27 17:37:46.511273 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:37:46.511247 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f9d7c6_94d4_4607_a5a6_1dbfa717f907.slice/crio-0756a377eef330e6016bba4734bad04f48a67a37cfc673af3a8c9c02566cd79e WatchSource:0}: Error finding container 0756a377eef330e6016bba4734bad04f48a67a37cfc673af3a8c9c02566cd79e: Status 404 returned error can't find the container with id 0756a377eef330e6016bba4734bad04f48a67a37cfc673af3a8c9c02566cd79e Jul 27 17:37:47.176767 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:47.176735 2582 generic.go:356] "Generic (PLEG): container finished" podID="41f9d7c6-94d4-4607-a5a6-1dbfa717f907" containerID="a807fb6ec4cfde6cc3bb7710a42b8405df91612556f66261aa4cbca93844bf7c" exitCode=0 Jul 27 17:37:47.177149 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:47.176819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" event={"ID":"41f9d7c6-94d4-4607-a5a6-1dbfa717f907","Type":"ContainerDied","Data":"a807fb6ec4cfde6cc3bb7710a42b8405df91612556f66261aa4cbca93844bf7c"} Jul 27 17:37:47.177149 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:47.176877 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" event={"ID":"41f9d7c6-94d4-4607-a5a6-1dbfa717f907","Type":"ContainerStarted","Data":"0756a377eef330e6016bba4734bad04f48a67a37cfc673af3a8c9c02566cd79e"} Jul 27 17:37:48.318584 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:48.318557 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:48.428962 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:48.428927 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f9d7c6-94d4-4607-a5a6-1dbfa717f907-maas-api-service-ca\") pod \"41f9d7c6-94d4-4607-a5a6-1dbfa717f907\" (UID: \"41f9d7c6-94d4-4607-a5a6-1dbfa717f907\") " Jul 27 17:37:48.429320 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:48.429297 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f9d7c6-94d4-4607-a5a6-1dbfa717f907-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41f9d7c6-94d4-4607-a5a6-1dbfa717f907" (UID: "41f9d7c6-94d4-4607-a5a6-1dbfa717f907"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:37:48.530644 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:48.530552 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41f9d7c6-94d4-4607-a5a6-1dbfa717f907-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:37:49.188952 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:49.188922 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" Jul 27 17:37:49.188952 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:49.188947 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj" event={"ID":"41f9d7c6-94d4-4607-a5a6-1dbfa717f907","Type":"ContainerDied","Data":"0756a377eef330e6016bba4734bad04f48a67a37cfc673af3a8c9c02566cd79e"} Jul 27 17:37:49.189161 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:37:49.188996 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0756a377eef330e6016bba4734bad04f48a67a37cfc673af3a8c9c02566cd79e" Jul 27 17:38:06.139921 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:06.139883 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj"] Jul 27 17:38:06.142363 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:06.142341 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f6e958-rdnlj"] Jul 27 17:38:06.199157 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:06.199112 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f9d7c6-94d4-4607-a5a6-1dbfa717f907" path="/var/lib/kubelet/pods/41f9d7c6-94d4-4607-a5a6-1dbfa717f907/volumes" Jul 27 17:38:13.167177 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.167134 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v"] Jul 27 17:38:13.167595 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.167581 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f9d7c6-94d4-4607-a5a6-1dbfa717f907" containerName="revoke-keys" Jul 27 17:38:13.167595 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.167596 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f9d7c6-94d4-4607-a5a6-1dbfa717f907" containerName="revoke-keys" Jul 27 17:38:13.167701 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.167690 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f9d7c6-94d4-4607-a5a6-1dbfa717f907" containerName="revoke-keys" Jul 27 17:38:13.170911 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.170894 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:13.173902 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.173880 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:38:13.178571 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.178444 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v"] Jul 27 17:38:13.268152 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.268102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c87402a5-592c-46f2-9f9b-61b192e17ee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-05e104-qbr5v\" (UID: \"c87402a5-592c-46f2-9f9b-61b192e17ee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:13.368989 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.368946 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c87402a5-592c-46f2-9f9b-61b192e17ee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-05e104-qbr5v\" (UID: \"c87402a5-592c-46f2-9f9b-61b192e17ee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:13.369643 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.369624 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c87402a5-592c-46f2-9f9b-61b192e17ee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-05e104-qbr5v\" (UID: \"c87402a5-592c-46f2-9f9b-61b192e17ee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:13.483678 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.483589 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:13.627179 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:13.627148 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v"] Jul 27 17:38:13.627673 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:38:13.627651 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc87402a5_592c_46f2_9f9b_61b192e17ee5.slice/crio-4d4abf2a0c0f7fe4ae86e87e570a159814c19541ed173feb9c446167d2352988 WatchSource:0}: Error finding container 4d4abf2a0c0f7fe4ae86e87e570a159814c19541ed173feb9c446167d2352988: Status 404 returned error can't find the container with id 4d4abf2a0c0f7fe4ae86e87e570a159814c19541ed173feb9c446167d2352988 Jul 27 17:38:14.335589 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:14.335554 2582 generic.go:356] "Generic (PLEG): container finished" podID="c87402a5-592c-46f2-9f9b-61b192e17ee5" containerID="bf281273aa423cc1850fda1c8551a3b2829abf51177c72d15cbd3fe332970f80" exitCode=0 Jul 27 17:38:14.335927 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:14.335596 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" event={"ID":"c87402a5-592c-46f2-9f9b-61b192e17ee5","Type":"ContainerDied","Data":"bf281273aa423cc1850fda1c8551a3b2829abf51177c72d15cbd3fe332970f80"} Jul 27 17:38:14.335927 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:14.335637 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" event={"ID":"c87402a5-592c-46f2-9f9b-61b192e17ee5","Type":"ContainerStarted","Data":"4d4abf2a0c0f7fe4ae86e87e570a159814c19541ed173feb9c446167d2352988"} Jul 27 17:38:15.481730 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:15.481705 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:15.589671 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:15.589631 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c87402a5-592c-46f2-9f9b-61b192e17ee5-maas-api-service-ca\") pod \"c87402a5-592c-46f2-9f9b-61b192e17ee5\" (UID: \"c87402a5-592c-46f2-9f9b-61b192e17ee5\") " Jul 27 17:38:15.590044 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:15.590017 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c87402a5-592c-46f2-9f9b-61b192e17ee5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c87402a5-592c-46f2-9f9b-61b192e17ee5" (UID: "c87402a5-592c-46f2-9f9b-61b192e17ee5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:38:15.690519 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:15.690435 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c87402a5-592c-46f2-9f9b-61b192e17ee5-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:38:16.351853 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:16.351825 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" Jul 27 17:38:16.351853 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:16.351841 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v" event={"ID":"c87402a5-592c-46f2-9f9b-61b192e17ee5","Type":"ContainerDied","Data":"4d4abf2a0c0f7fe4ae86e87e570a159814c19541ed173feb9c446167d2352988"} Jul 27 17:38:16.352053 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:16.351886 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d4abf2a0c0f7fe4ae86e87e570a159814c19541ed173feb9c446167d2352988" Jul 27 17:38:33.274143 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:33.274100 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v"] Jul 27 17:38:33.276509 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:33.276487 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-05e104-qbr5v"] Jul 27 17:38:34.199360 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:38:34.199328 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c87402a5-592c-46f2-9f9b-61b192e17ee5" path="/var/lib/kubelet/pods/c87402a5-592c-46f2-9f9b-61b192e17ee5/volumes" Jul 27 17:39:40.480134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:39:40.480097 2582 scope.go:117] "RemoveContainer" containerID="65a45ac66c6598935ec4d6cfad54147b32bc5c9a456b5c89eed935ece52c53d3" Jul 27 17:42:40.519152 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:42:40.519116 2582 scope.go:117] "RemoveContainer" containerID="aadedd9bb8d31c37e8b264d9fe80f57c4f25583093097351bbb7884b30cec18b" Jul 27 17:43:40.540988 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:43:40.540952 2582 scope.go:117] "RemoveContainer" containerID="f0536ddc8364d9e7b5d286b1cd401100023d2907f4fcbe894b3fe8ee6690c8a0" Jul 27 17:44:40.565410 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:44:40.565364 2582 scope.go:117] "RemoveContainer" containerID="bf281273aa423cc1850fda1c8551a3b2829abf51177c72d15cbd3fe332970f80" Jul 27 17:44:40.574431 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:44:40.574406 2582 scope.go:117] "RemoveContainer" containerID="a807fb6ec4cfde6cc3bb7710a42b8405df91612556f66261aa4cbca93844bf7c" Jul 27 17:45:00.134579 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.134543 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh"] Jul 27 17:45:00.135186 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.135165 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c87402a5-592c-46f2-9f9b-61b192e17ee5" containerName="revoke-keys" Jul 27 17:45:00.135279 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.135190 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c87402a5-592c-46f2-9f9b-61b192e17ee5" containerName="revoke-keys" Jul 27 17:45:00.135333 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.135287 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c87402a5-592c-46f2-9f9b-61b192e17ee5" containerName="revoke-keys" Jul 27 17:45:00.141638 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.141601 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss"] Jul 27 17:45:00.143518 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.143490 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" Jul 27 17:45:00.147308 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.147026 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps"] Jul 27 17:45:00.147308 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.147055 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:45:00.147549 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.147364 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" Jul 27 17:45:00.153444 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.153417 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh"] Jul 27 17:45:00.153605 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.153587 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" Jul 27 17:45:00.156235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.156219 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss"] Jul 27 17:45:00.158400 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.158362 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps"] Jul 27 17:45:00.161157 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.161141 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" Jul 27 17:45:00.170326 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.170304 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" Jul 27 17:45:00.210869 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.210840 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" Jul 27 17:45:00.391779 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.391752 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps"] Jul 27 17:45:00.392896 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:45:00.392873 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826db24b_39e0_4eca_8623_15fec7b43ce7.slice/crio-f106b07ab5018bcb9b773ff8d922b123cfaf689d34cb9bbde038acf16b4c8f7e WatchSource:0}: Error finding container f106b07ab5018bcb9b773ff8d922b123cfaf689d34cb9bbde038acf16b4c8f7e: Status 404 returned error can't find the container with id f106b07ab5018bcb9b773ff8d922b123cfaf689d34cb9bbde038acf16b4c8f7e Jul 27 17:45:00.539390 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.539357 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh"] Jul 27 17:45:00.540432 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:45:00.540400 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1990f98_81d9_4fa4_995c_dc29438ada8c.slice/crio-be8acefc963df24be587b2562c99853848c42747f42296d7f96f6920c3e83548 WatchSource:0}: Error finding container be8acefc963df24be587b2562c99853848c42747f42296d7f96f6920c3e83548: Status 404 returned error can't find the container with id be8acefc963df24be587b2562c99853848c42747f42296d7f96f6920c3e83548 Jul 27 17:45:00.566014 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.565988 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss"] Jul 27 17:45:00.574534 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:45:00.574511 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d2b3a8_ee85_4412_8fa2_3127f228562f.slice/crio-b659acf338332357f8adea6952369e55e5d40c5f95039c2277ec88b29efbccf6 WatchSource:0}: Error finding container b659acf338332357f8adea6952369e55e5d40c5f95039c2277ec88b29efbccf6: Status 404 returned error can't find the container with id b659acf338332357f8adea6952369e55e5d40c5f95039c2277ec88b29efbccf6 Jul 27 17:45:00.732444 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.732411 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" event={"ID":"f1d2b3a8-ee85-4412-8fa2-3127f228562f","Type":"ContainerStarted","Data":"b659acf338332357f8adea6952369e55e5d40c5f95039c2277ec88b29efbccf6"} Jul 27 17:45:00.733653 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.733632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" event={"ID":"b1990f98-81d9-4fa4-995c-dc29438ada8c","Type":"ContainerStarted","Data":"be8acefc963df24be587b2562c99853848c42747f42296d7f96f6920c3e83548"} Jul 27 17:45:00.735082 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.735062 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" event={"ID":"826db24b-39e0-4eca-8623-15fec7b43ce7","Type":"ContainerStarted","Data":"e00cf88c805c8456965f5af28262af3f13ba06793ff70c53e049d5fc3278f38a"} Jul 27 17:45:00.735158 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.735105 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" event={"ID":"826db24b-39e0-4eca-8623-15fec7b43ce7","Type":"ContainerStarted","Data":"f106b07ab5018bcb9b773ff8d922b123cfaf689d34cb9bbde038acf16b4c8f7e"} Jul 27 17:45:00.751723 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:00.751673 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" podStartSLOduration=0.751659458 podStartE2EDuration="751.659458ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:45:00.750188687 +0000 UTC m=+2907.034347971" watchObservedRunningTime="2026-07-27 17:45:00.751659458 +0000 UTC m=+2907.035818740" Jul 27 17:45:01.742682 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:01.742643 2582 generic.go:356] "Generic (PLEG): container finished" podID="b1990f98-81d9-4fa4-995c-dc29438ada8c" containerID="e0cedc917e91cdbc432c08f994e918af5e404a2b52630c0391c21084fb6128dd" exitCode=0 Jul 27 17:45:01.743079 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:01.742729 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" event={"ID":"b1990f98-81d9-4fa4-995c-dc29438ada8c","Type":"ContainerDied","Data":"e0cedc917e91cdbc432c08f994e918af5e404a2b52630c0391c21084fb6128dd"} Jul 27 17:45:01.744127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:01.744107 2582 generic.go:356] "Generic (PLEG): container finished" podID="826db24b-39e0-4eca-8623-15fec7b43ce7" containerID="e00cf88c805c8456965f5af28262af3f13ba06793ff70c53e049d5fc3278f38a" exitCode=0 Jul 27 17:45:01.744243 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:01.744189 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" event={"ID":"826db24b-39e0-4eca-8623-15fec7b43ce7","Type":"ContainerDied","Data":"e00cf88c805c8456965f5af28262af3f13ba06793ff70c53e049d5fc3278f38a"} Jul 27 17:45:01.745723 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:01.745702 2582 generic.go:356] "Generic (PLEG): container finished" podID="f1d2b3a8-ee85-4412-8fa2-3127f228562f" containerID="897dd7186d841f6d683cb50e63fc1384ce49bc71d3f62b8943a981d10778addd" exitCode=0 Jul 27 17:45:01.745814 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:01.745723 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" event={"ID":"f1d2b3a8-ee85-4412-8fa2-3127f228562f","Type":"ContainerDied","Data":"897dd7186d841f6d683cb50e63fc1384ce49bc71d3f62b8943a981d10778addd"} Jul 27 17:45:02.916465 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:02.916436 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" Jul 27 17:45:02.958528 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:02.958500 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" Jul 27 17:45:02.961262 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:02.961240 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" Jul 27 17:45:03.765190 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.765161 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" Jul 27 17:45:03.765367 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.765202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss" event={"ID":"f1d2b3a8-ee85-4412-8fa2-3127f228562f","Type":"ContainerDied","Data":"b659acf338332357f8adea6952369e55e5d40c5f95039c2277ec88b29efbccf6"} Jul 27 17:45:03.765367 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.765287 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b659acf338332357f8adea6952369e55e5d40c5f95039c2277ec88b29efbccf6" Jul 27 17:45:03.766671 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.766646 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" Jul 27 17:45:03.766803 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.766665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh" event={"ID":"b1990f98-81d9-4fa4-995c-dc29438ada8c","Type":"ContainerDied","Data":"be8acefc963df24be587b2562c99853848c42747f42296d7f96f6920c3e83548"} Jul 27 17:45:03.766803 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.766723 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be8acefc963df24be587b2562c99853848c42747f42296d7f96f6920c3e83548" Jul 27 17:45:03.768066 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.768036 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" event={"ID":"826db24b-39e0-4eca-8623-15fec7b43ce7","Type":"ContainerDied","Data":"f106b07ab5018bcb9b773ff8d922b123cfaf689d34cb9bbde038acf16b4c8f7e"} Jul 27 17:45:03.768161 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.768076 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f106b07ab5018bcb9b773ff8d922b123cfaf689d34cb9bbde038acf16b4c8f7e" Jul 27 17:45:03.768161 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:45:03.768046 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29752905-pb7ps" Jul 27 17:47:44.659092 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659051 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-139766024846768"] Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659498 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1990f98-81d9-4fa4-995c-dc29438ada8c" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659511 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1990f98-81d9-4fa4-995c-dc29438ada8c" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659529 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="826db24b-39e0-4eca-8623-15fec7b43ce7" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659534 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="826db24b-39e0-4eca-8623-15fec7b43ce7" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659547 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1d2b3a8-ee85-4412-8fa2-3127f228562f" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659552 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d2b3a8-ee85-4412-8fa2-3127f228562f" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659610 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1990f98-81d9-4fa4-995c-dc29438ada8c" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659620 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="826db24b-39e0-4eca-8623-15fec7b43ce7" containerName="cleanup" Jul 27 17:47:44.661566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.659627 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1d2b3a8-ee85-4412-8fa2-3127f228562f" containerName="cleanup" Jul 27 17:47:44.662643 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.662625 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:44.667160 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.667136 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-k9wh9\"" Jul 27 17:47:44.672025 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.671997 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766024846768"] Jul 27 17:47:44.797445 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.797407 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvlcl\" (UniqueName: \"kubernetes.io/projected/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7-kube-api-access-cvlcl\") pod \"test-curl-7411-139766024846768\" (UID: \"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7\") " pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:44.898714 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.898675 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlcl\" (UniqueName: \"kubernetes.io/projected/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7-kube-api-access-cvlcl\") pod \"test-curl-7411-139766024846768\" (UID: \"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7\") " pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:44.910327 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.910255 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvlcl\" (UniqueName: \"kubernetes.io/projected/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7-kube-api-access-cvlcl\") pod \"test-curl-7411-139766024846768\" (UID: \"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7\") " pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:44.972250 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:44.972216 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:45.120392 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:45.120353 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766024846768"] Jul 27 17:47:45.122423 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:47:45.122396 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40c5d403_c9ab_4c10_994b_ed5bdb3b99e7.slice/crio-8453d13dc54ed20e308ba6aee595887cdd0692efadc1e70221bf2b49519c246d WatchSource:0}: Error finding container 8453d13dc54ed20e308ba6aee595887cdd0692efadc1e70221bf2b49519c246d: Status 404 returned error can't find the container with id 8453d13dc54ed20e308ba6aee595887cdd0692efadc1e70221bf2b49519c246d Jul 27 17:47:45.123842 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:45.123822 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 17:47:45.710948 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:45.710909 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766024846768" event={"ID":"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7","Type":"ContainerStarted","Data":"8453d13dc54ed20e308ba6aee595887cdd0692efadc1e70221bf2b49519c246d"} Jul 27 17:47:46.718288 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:46.718253 2582 generic.go:356] "Generic (PLEG): container finished" podID="40c5d403-c9ab-4c10-994b-ed5bdb3b99e7" containerID="98148ec157bbc648243b5b17f3628f8bd0bc711c2d601dfa8cecddfd11cfb6be" exitCode=0 Jul 27 17:47:46.718288 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:46.718292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766024846768" event={"ID":"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7","Type":"ContainerDied","Data":"98148ec157bbc648243b5b17f3628f8bd0bc711c2d601dfa8cecddfd11cfb6be"} Jul 27 17:47:47.864086 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:47.864058 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:47.905715 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:47.905686 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-139766024846768_40c5d403-c9ab-4c10-994b-ed5bdb3b99e7/test-curl-7411-139766024846768/0.log" Jul 27 17:47:47.935726 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:47.935692 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-139766024846768"] Jul 27 17:47:47.941422 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:47.941388 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-139766024846768"] Jul 27 17:47:48.037564 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.037468 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvlcl\" (UniqueName: \"kubernetes.io/projected/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7-kube-api-access-cvlcl\") pod \"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7\" (UID: \"40c5d403-c9ab-4c10-994b-ed5bdb3b99e7\") " Jul 27 17:47:48.039939 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.039907 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7-kube-api-access-cvlcl" (OuterVolumeSpecName: "kube-api-access-cvlcl") pod "40c5d403-c9ab-4c10-994b-ed5bdb3b99e7" (UID: "40c5d403-c9ab-4c10-994b-ed5bdb3b99e7"). InnerVolumeSpecName "kube-api-access-cvlcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:47:48.058337 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.058302 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-139766023946288"] Jul 27 17:47:48.058821 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.058804 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c5d403-c9ab-4c10-994b-ed5bdb3b99e7" containerName="test-curl-7411-139766024846768" Jul 27 17:47:48.058868 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.058822 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c5d403-c9ab-4c10-994b-ed5bdb3b99e7" containerName="test-curl-7411-139766024846768" Jul 27 17:47:48.058904 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.058890 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c5d403-c9ab-4c10-994b-ed5bdb3b99e7" containerName="test-curl-7411-139766024846768" Jul 27 17:47:48.062058 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.062043 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:48.069509 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.069479 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766023946288"] Jul 27 17:47:48.138488 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.138452 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvlcl\" (UniqueName: \"kubernetes.io/projected/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7-kube-api-access-cvlcl\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:47:48.198728 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.198696 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c5d403-c9ab-4c10-994b-ed5bdb3b99e7" path="/var/lib/kubelet/pods/40c5d403-c9ab-4c10-994b-ed5bdb3b99e7/volumes" Jul 27 17:47:48.239419 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.239363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltxdb\" (UniqueName: \"kubernetes.io/projected/f0096717-7c98-486a-8559-34700f1a9066-kube-api-access-ltxdb\") pod \"test-curl-7411-139766023946288\" (UID: \"f0096717-7c98-486a-8559-34700f1a9066\") " pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:48.340350 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.340314 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxdb\" (UniqueName: \"kubernetes.io/projected/f0096717-7c98-486a-8559-34700f1a9066-kube-api-access-ltxdb\") pod \"test-curl-7411-139766023946288\" (UID: \"f0096717-7c98-486a-8559-34700f1a9066\") " pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:48.350469 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.350449 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltxdb\" (UniqueName: \"kubernetes.io/projected/f0096717-7c98-486a-8559-34700f1a9066-kube-api-access-ltxdb\") pod \"test-curl-7411-139766023946288\" (UID: \"f0096717-7c98-486a-8559-34700f1a9066\") " pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:48.373340 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.373308 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:48.514386 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.514348 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766023946288"] Jul 27 17:47:48.515768 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:47:48.515747 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0096717_7c98_486a_8559_34700f1a9066.slice/crio-9de940b3bc174e0cd656433a79050f306d374aea19fb6212cc6b2996c75010b4 WatchSource:0}: Error finding container 9de940b3bc174e0cd656433a79050f306d374aea19fb6212cc6b2996c75010b4: Status 404 returned error can't find the container with id 9de940b3bc174e0cd656433a79050f306d374aea19fb6212cc6b2996c75010b4 Jul 27 17:47:48.730864 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.730772 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024846768" Jul 27 17:47:48.731025 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.730776 2582 scope.go:117] "RemoveContainer" containerID="98148ec157bbc648243b5b17f3628f8bd0bc711c2d601dfa8cecddfd11cfb6be" Jul 27 17:47:48.736343 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:48.736298 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766023946288" event={"ID":"f0096717-7c98-486a-8559-34700f1a9066","Type":"ContainerStarted","Data":"9de940b3bc174e0cd656433a79050f306d374aea19fb6212cc6b2996c75010b4"} Jul 27 17:47:49.745023 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:49.744988 2582 generic.go:356] "Generic (PLEG): container finished" podID="f0096717-7c98-486a-8559-34700f1a9066" containerID="ab875226f0ba9350b21c37aec4e6c8d69341f8ee45b701c91ee694f57440be75" exitCode=0 Jul 27 17:47:49.745464 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:49.745084 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766023946288" event={"ID":"f0096717-7c98-486a-8559-34700f1a9066","Type":"ContainerDied","Data":"ab875226f0ba9350b21c37aec4e6c8d69341f8ee45b701c91ee694f57440be75"} Jul 27 17:47:50.895668 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:50.895640 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:50.920132 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:50.920107 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-139766023946288_f0096717-7c98-486a-8559-34700f1a9066/test-curl-7411-139766023946288/0.log" Jul 27 17:47:50.946880 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:50.946845 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-139766023946288"] Jul 27 17:47:50.951291 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:50.951263 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-139766023946288"] Jul 27 17:47:50.966816 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:50.966791 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltxdb\" (UniqueName: \"kubernetes.io/projected/f0096717-7c98-486a-8559-34700f1a9066-kube-api-access-ltxdb\") pod \"f0096717-7c98-486a-8559-34700f1a9066\" (UID: \"f0096717-7c98-486a-8559-34700f1a9066\") " Jul 27 17:47:50.968943 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:50.968920 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0096717-7c98-486a-8559-34700f1a9066-kube-api-access-ltxdb" (OuterVolumeSpecName: "kube-api-access-ltxdb") pod "f0096717-7c98-486a-8559-34700f1a9066" (UID: "f0096717-7c98-486a-8559-34700f1a9066"). InnerVolumeSpecName "kube-api-access-ltxdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:47:51.068354 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.068264 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltxdb\" (UniqueName: \"kubernetes.io/projected/f0096717-7c98-486a-8559-34700f1a9066-kube-api-access-ltxdb\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:47:51.103992 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.103957 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:51.104374 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.104361 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0096717-7c98-486a-8559-34700f1a9066" containerName="test-curl-7411-139766023946288" Jul 27 17:47:51.104374 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.104389 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0096717-7c98-486a-8559-34700f1a9066" containerName="test-curl-7411-139766023946288" Jul 27 17:47:51.104502 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.104472 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0096717-7c98-486a-8559-34700f1a9066" containerName="test-curl-7411-139766023946288" Jul 27 17:47:51.107567 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.107551 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:51.116983 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.116890 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:51.169181 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.169140 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc5d7\" (UniqueName: \"kubernetes.io/projected/6e91b1ad-67c4-4611-8740-58da367f0aee-kube-api-access-vc5d7\") pod \"test-curl-7411-139766024871632\" (UID: \"6e91b1ad-67c4-4611-8740-58da367f0aee\") " pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:51.270514 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.270477 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc5d7\" (UniqueName: \"kubernetes.io/projected/6e91b1ad-67c4-4611-8740-58da367f0aee-kube-api-access-vc5d7\") pod \"test-curl-7411-139766024871632\" (UID: \"6e91b1ad-67c4-4611-8740-58da367f0aee\") " pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:51.281025 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.280993 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc5d7\" (UniqueName: \"kubernetes.io/projected/6e91b1ad-67c4-4611-8740-58da367f0aee-kube-api-access-vc5d7\") pod \"test-curl-7411-139766024871632\" (UID: \"6e91b1ad-67c4-4611-8740-58da367f0aee\") " pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:51.418364 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.418330 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:51.558972 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.558942 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:51.560022 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:47:51.559997 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e91b1ad_67c4_4611_8740_58da367f0aee.slice/crio-549c7d5f3760be6d5f5817db7811456a0512a80d2862d1c318df9c19fb8e5e52 WatchSource:0}: Error finding container 549c7d5f3760be6d5f5817db7811456a0512a80d2862d1c318df9c19fb8e5e52: Status 404 returned error can't find the container with id 549c7d5f3760be6d5f5817db7811456a0512a80d2862d1c318df9c19fb8e5e52 Jul 27 17:47:51.757896 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.757799 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766024871632" event={"ID":"6e91b1ad-67c4-4611-8740-58da367f0aee","Type":"ContainerStarted","Data":"549c7d5f3760be6d5f5817db7811456a0512a80d2862d1c318df9c19fb8e5e52"} Jul 27 17:47:51.759109 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.759089 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:51.759219 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:51.759112 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9de940b3bc174e0cd656433a79050f306d374aea19fb6212cc6b2996c75010b4" Jul 27 17:47:51.772164 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:47:51.772133 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-139766023946288\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="f0096717-7c98-486a-8559-34700f1a9066" pod="opendatahub/test-curl-7411-139766023946288" Jul 27 17:47:52.199458 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:52.199425 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0096717-7c98-486a-8559-34700f1a9066" path="/var/lib/kubelet/pods/f0096717-7c98-486a-8559-34700f1a9066/volumes" Jul 27 17:47:52.765994 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:52.765961 2582 generic.go:356] "Generic (PLEG): container finished" podID="6e91b1ad-67c4-4611-8740-58da367f0aee" containerID="666e753846391d1e6fe05725ad7ca9bc1c999b820c27c78dabab8f579b4c755e" exitCode=0 Jul 27 17:47:52.766186 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:52.766045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766024871632" event={"ID":"6e91b1ad-67c4-4611-8740-58da367f0aee","Type":"ContainerDied","Data":"666e753846391d1e6fe05725ad7ca9bc1c999b820c27c78dabab8f579b4c755e"} Jul 27 17:47:53.910183 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:53.910155 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:53.935135 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:53.935107 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-139766024871632_6e91b1ad-67c4-4611-8740-58da367f0aee/test-curl-7411-139766024871632/0.log" Jul 27 17:47:53.958458 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:53.958416 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:53.961693 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:53.961668 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:53.998584 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:53.998549 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc5d7\" (UniqueName: \"kubernetes.io/projected/6e91b1ad-67c4-4611-8740-58da367f0aee-kube-api-access-vc5d7\") pod \"6e91b1ad-67c4-4611-8740-58da367f0aee\" (UID: \"6e91b1ad-67c4-4611-8740-58da367f0aee\") " Jul 27 17:47:54.000666 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.000643 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e91b1ad-67c4-4611-8740-58da367f0aee-kube-api-access-vc5d7" (OuterVolumeSpecName: "kube-api-access-vc5d7") pod "6e91b1ad-67c4-4611-8740-58da367f0aee" (UID: "6e91b1ad-67c4-4611-8740-58da367f0aee"). InnerVolumeSpecName "kube-api-access-vc5d7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:47:54.085494 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.085460 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:54.085965 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.085947 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e91b1ad-67c4-4611-8740-58da367f0aee" containerName="test-curl-7411-139766024871632" Jul 27 17:47:54.086053 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.085969 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e91b1ad-67c4-4611-8740-58da367f0aee" containerName="test-curl-7411-139766024871632" Jul 27 17:47:54.086117 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.086077 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e91b1ad-67c4-4611-8740-58da367f0aee" containerName="test-curl-7411-139766024871632" Jul 27 17:47:54.089247 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.089229 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:54.095130 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.095103 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:54.099311 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.099292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9hhh\" (UniqueName: \"kubernetes.io/projected/b59e6961-6979-4bb4-b9d4-3612c8184913-kube-api-access-d9hhh\") pod \"test-curl-7411-139766024871632\" (UID: \"b59e6961-6979-4bb4-b9d4-3612c8184913\") " pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:54.099436 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.099345 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc5d7\" (UniqueName: \"kubernetes.io/projected/6e91b1ad-67c4-4611-8740-58da367f0aee-kube-api-access-vc5d7\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:47:54.198793 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.198762 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e91b1ad-67c4-4611-8740-58da367f0aee" path="/var/lib/kubelet/pods/6e91b1ad-67c4-4611-8740-58da367f0aee/volumes" Jul 27 17:47:54.199733 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.199713 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9hhh\" (UniqueName: \"kubernetes.io/projected/b59e6961-6979-4bb4-b9d4-3612c8184913-kube-api-access-d9hhh\") pod \"test-curl-7411-139766024871632\" (UID: \"b59e6961-6979-4bb4-b9d4-3612c8184913\") " pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:54.212330 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.212306 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9hhh\" (UniqueName: \"kubernetes.io/projected/b59e6961-6979-4bb4-b9d4-3612c8184913-kube-api-access-d9hhh\") pod \"test-curl-7411-139766024871632\" (UID: \"b59e6961-6979-4bb4-b9d4-3612c8184913\") " pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:54.399599 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.399502 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:54.542487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.542457 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:54.544015 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:47:54.543991 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb59e6961_6979_4bb4_b9d4_3612c8184913.slice/crio-99c35b1e23fb19123b1473f58a85ec4fd2df12d26f28518f9416063ff9c859fa WatchSource:0}: Error finding container 99c35b1e23fb19123b1473f58a85ec4fd2df12d26f28518f9416063ff9c859fa: Status 404 returned error can't find the container with id 99c35b1e23fb19123b1473f58a85ec4fd2df12d26f28518f9416063ff9c859fa Jul 27 17:47:54.780259 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.780182 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:54.780259 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.780235 2582 scope.go:117] "RemoveContainer" containerID="666e753846391d1e6fe05725ad7ca9bc1c999b820c27c78dabab8f579b4c755e" Jul 27 17:47:54.781750 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:54.781728 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766024871632" event={"ID":"b59e6961-6979-4bb4-b9d4-3612c8184913","Type":"ContainerStarted","Data":"99c35b1e23fb19123b1473f58a85ec4fd2df12d26f28518f9416063ff9c859fa"} Jul 27 17:47:55.790242 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:55.790210 2582 generic.go:356] "Generic (PLEG): container finished" podID="b59e6961-6979-4bb4-b9d4-3612c8184913" containerID="f4c9555f02be3718e3bd6e2443c3f1ebe0ead0c50f5821dd638e4ff98d1fe6d2" exitCode=0 Jul 27 17:47:55.790663 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:55.790299 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766024871632" event={"ID":"b59e6961-6979-4bb4-b9d4-3612c8184913","Type":"ContainerDied","Data":"f4c9555f02be3718e3bd6e2443c3f1ebe0ead0c50f5821dd638e4ff98d1fe6d2"} Jul 27 17:47:56.962039 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:56.962016 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:56.995010 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:56.994977 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-139766024871632_b59e6961-6979-4bb4-b9d4-3612c8184913/test-curl-7411-139766024871632/0.log" Jul 27 17:47:57.023255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.023217 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:57.026798 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.026772 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-139766024871632"] Jul 27 17:47:57.126260 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.126230 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9hhh\" (UniqueName: \"kubernetes.io/projected/b59e6961-6979-4bb4-b9d4-3612c8184913-kube-api-access-d9hhh\") pod \"b59e6961-6979-4bb4-b9d4-3612c8184913\" (UID: \"b59e6961-6979-4bb4-b9d4-3612c8184913\") " Jul 27 17:47:57.128330 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.128308 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59e6961-6979-4bb4-b9d4-3612c8184913-kube-api-access-d9hhh" (OuterVolumeSpecName: "kube-api-access-d9hhh") pod "b59e6961-6979-4bb4-b9d4-3612c8184913" (UID: "b59e6961-6979-4bb4-b9d4-3612c8184913"). InnerVolumeSpecName "kube-api-access-d9hhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:47:57.152400 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.152342 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:47:57.152813 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.152800 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b59e6961-6979-4bb4-b9d4-3612c8184913" containerName="test-curl-7411-139766024871632" Jul 27 17:47:57.152872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.152815 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b59e6961-6979-4bb4-b9d4-3612c8184913" containerName="test-curl-7411-139766024871632" Jul 27 17:47:57.152905 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.152892 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b59e6961-6979-4bb4-b9d4-3612c8184913" containerName="test-curl-7411-139766024871632" Jul 27 17:47:57.155940 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.155921 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:47:57.168281 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.168254 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:47:57.227660 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.227625 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fslzq\" (UniqueName: \"kubernetes.io/projected/038973d0-4b93-4ebf-9350-2d6f859bb79c-kube-api-access-fslzq\") pod \"test-curl-7411-139766023919200\" (UID: \"038973d0-4b93-4ebf-9350-2d6f859bb79c\") " pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:47:57.227862 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.227794 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9hhh\" (UniqueName: \"kubernetes.io/projected/b59e6961-6979-4bb4-b9d4-3612c8184913-kube-api-access-d9hhh\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:47:57.329246 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.329205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fslzq\" (UniqueName: \"kubernetes.io/projected/038973d0-4b93-4ebf-9350-2d6f859bb79c-kube-api-access-fslzq\") pod \"test-curl-7411-139766023919200\" (UID: \"038973d0-4b93-4ebf-9350-2d6f859bb79c\") " pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:47:57.339820 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.339794 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fslzq\" (UniqueName: \"kubernetes.io/projected/038973d0-4b93-4ebf-9350-2d6f859bb79c-kube-api-access-fslzq\") pod \"test-curl-7411-139766023919200\" (UID: \"038973d0-4b93-4ebf-9350-2d6f859bb79c\") " pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:47:57.465354 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.465265 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:47:57.805653 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.805577 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:57.805653 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.805618 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99c35b1e23fb19123b1473f58a85ec4fd2df12d26f28518f9416063ff9c859fa" Jul 27 17:47:57.816596 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:57.816572 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:47:57.817139 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:47:57.817113 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-139766024871632\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="b59e6961-6979-4bb4-b9d4-3612c8184913" pod="opendatahub/test-curl-7411-139766024871632" Jul 27 17:47:57.818358 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:47:57.818333 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod038973d0_4b93_4ebf_9350_2d6f859bb79c.slice/crio-84b99588fac802c501b77d3754380d0e2f27428f4869ffdd12163ae0702ed902 WatchSource:0}: Error finding container 84b99588fac802c501b77d3754380d0e2f27428f4869ffdd12163ae0702ed902: Status 404 returned error can't find the container with id 84b99588fac802c501b77d3754380d0e2f27428f4869ffdd12163ae0702ed902 Jul 27 17:47:58.199335 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:58.199308 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b59e6961-6979-4bb4-b9d4-3612c8184913" path="/var/lib/kubelet/pods/b59e6961-6979-4bb4-b9d4-3612c8184913/volumes" Jul 27 17:47:58.813371 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:58.813338 2582 generic.go:356] "Generic (PLEG): container finished" podID="038973d0-4b93-4ebf-9350-2d6f859bb79c" containerID="a92548eb8c9b1e27693794d81218f8db9fa328ecd802dcc4d667d5dfbe872ac0" exitCode=0 Jul 27 17:47:58.813539 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:58.813406 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766023919200" event={"ID":"038973d0-4b93-4ebf-9350-2d6f859bb79c","Type":"ContainerDied","Data":"a92548eb8c9b1e27693794d81218f8db9fa328ecd802dcc4d667d5dfbe872ac0"} Jul 27 17:47:58.813539 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:58.813441 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766023919200" event={"ID":"038973d0-4b93-4ebf-9350-2d6f859bb79c","Type":"ContainerStarted","Data":"84b99588fac802c501b77d3754380d0e2f27428f4869ffdd12163ae0702ed902"} Jul 27 17:47:59.961532 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:59.961472 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:47:59.985699 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:47:59.985670 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-139766023919200_038973d0-4b93-4ebf-9350-2d6f859bb79c/test-curl-7411-139766023919200/0.log" Jul 27 17:48:00.020406 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.020356 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:00.026501 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.026474 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:00.054123 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.054094 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fslzq\" (UniqueName: \"kubernetes.io/projected/038973d0-4b93-4ebf-9350-2d6f859bb79c-kube-api-access-fslzq\") pod \"038973d0-4b93-4ebf-9350-2d6f859bb79c\" (UID: \"038973d0-4b93-4ebf-9350-2d6f859bb79c\") " Jul 27 17:48:00.056191 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.056168 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038973d0-4b93-4ebf-9350-2d6f859bb79c-kube-api-access-fslzq" (OuterVolumeSpecName: "kube-api-access-fslzq") pod "038973d0-4b93-4ebf-9350-2d6f859bb79c" (UID: "038973d0-4b93-4ebf-9350-2d6f859bb79c"). InnerVolumeSpecName "kube-api-access-fslzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:48:00.154887 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.154848 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:00.155244 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.155223 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fslzq\" (UniqueName: \"kubernetes.io/projected/038973d0-4b93-4ebf-9350-2d6f859bb79c-kube-api-access-fslzq\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:48:00.155284 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.155264 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038973d0-4b93-4ebf-9350-2d6f859bb79c" containerName="test-curl-7411-139766023919200" Jul 27 17:48:00.155284 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.155276 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="038973d0-4b93-4ebf-9350-2d6f859bb79c" containerName="test-curl-7411-139766023919200" Jul 27 17:48:00.155372 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.155362 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="038973d0-4b93-4ebf-9350-2d6f859bb79c" containerName="test-curl-7411-139766023919200" Jul 27 17:48:00.158386 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.158365 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:00.171861 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.171833 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:00.200573 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.200545 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="038973d0-4b93-4ebf-9350-2d6f859bb79c" path="/var/lib/kubelet/pods/038973d0-4b93-4ebf-9350-2d6f859bb79c/volumes" Jul 27 17:48:00.256007 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.255924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g9zl\" (UniqueName: \"kubernetes.io/projected/01c35c51-e858-4269-b4b2-e7303cdeeda2-kube-api-access-5g9zl\") pod \"test-curl-7411-139766023919200\" (UID: \"01c35c51-e858-4269-b4b2-e7303cdeeda2\") " pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:00.356726 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.356692 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g9zl\" (UniqueName: \"kubernetes.io/projected/01c35c51-e858-4269-b4b2-e7303cdeeda2-kube-api-access-5g9zl\") pod \"test-curl-7411-139766023919200\" (UID: \"01c35c51-e858-4269-b4b2-e7303cdeeda2\") " pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:00.367230 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.367202 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g9zl\" (UniqueName: \"kubernetes.io/projected/01c35c51-e858-4269-b4b2-e7303cdeeda2-kube-api-access-5g9zl\") pod \"test-curl-7411-139766023919200\" (UID: \"01c35c51-e858-4269-b4b2-e7303cdeeda2\") " pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:00.467433 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.467397 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:00.616843 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.616816 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:00.617556 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:48:00.617531 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01c35c51_e858_4269_b4b2_e7303cdeeda2.slice/crio-fe58a2ca06d9a5a94c157249ef17ee3a6b319ff7b427f7ddeecba1bbf68d6e4b WatchSource:0}: Error finding container fe58a2ca06d9a5a94c157249ef17ee3a6b319ff7b427f7ddeecba1bbf68d6e4b: Status 404 returned error can't find the container with id fe58a2ca06d9a5a94c157249ef17ee3a6b319ff7b427f7ddeecba1bbf68d6e4b Jul 27 17:48:00.833873 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.833788 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:00.833873 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.833806 2582 scope.go:117] "RemoveContainer" containerID="a92548eb8c9b1e27693794d81218f8db9fa328ecd802dcc4d667d5dfbe872ac0" Jul 27 17:48:00.835200 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:00.835002 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766023919200" event={"ID":"01c35c51-e858-4269-b4b2-e7303cdeeda2","Type":"ContainerStarted","Data":"fe58a2ca06d9a5a94c157249ef17ee3a6b319ff7b427f7ddeecba1bbf68d6e4b"} Jul 27 17:48:01.843970 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:01.843926 2582 generic.go:356] "Generic (PLEG): container finished" podID="01c35c51-e858-4269-b4b2-e7303cdeeda2" containerID="610670c79e9fd0cfb4691ee9b8b9b303459d646dbb7d2b0993d213bed6aaa746" exitCode=0 Jul 27 17:48:01.844435 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:01.843983 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-139766023919200" event={"ID":"01c35c51-e858-4269-b4b2-e7303cdeeda2","Type":"ContainerDied","Data":"610670c79e9fd0cfb4691ee9b8b9b303459d646dbb7d2b0993d213bed6aaa746"} Jul 27 17:48:02.996998 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:02.996969 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:03.023679 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.023649 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-139766023919200_01c35c51-e858-4269-b4b2-e7303cdeeda2/test-curl-7411-139766023919200/0.log" Jul 27 17:48:03.055494 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.055450 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:03.061566 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.061535 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-139766023919200"] Jul 27 17:48:03.082252 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.082221 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g9zl\" (UniqueName: \"kubernetes.io/projected/01c35c51-e858-4269-b4b2-e7303cdeeda2-kube-api-access-5g9zl\") pod \"01c35c51-e858-4269-b4b2-e7303cdeeda2\" (UID: \"01c35c51-e858-4269-b4b2-e7303cdeeda2\") " Jul 27 17:48:03.084323 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.084297 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c35c51-e858-4269-b4b2-e7303cdeeda2-kube-api-access-5g9zl" (OuterVolumeSpecName: "kube-api-access-5g9zl") pod "01c35c51-e858-4269-b4b2-e7303cdeeda2" (UID: "01c35c51-e858-4269-b4b2-e7303cdeeda2"). InnerVolumeSpecName "kube-api-access-5g9zl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:48:03.183005 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.182914 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g9zl\" (UniqueName: \"kubernetes.io/projected/01c35c51-e858-4269-b4b2-e7303cdeeda2-kube-api-access-5g9zl\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:48:03.857614 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.857583 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe58a2ca06d9a5a94c157249ef17ee3a6b319ff7b427f7ddeecba1bbf68d6e4b" Jul 27 17:48:03.857614 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:03.857617 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:03.869591 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:48:03.869563 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-139766023919200\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:04.067830 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.067796 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26"] Jul 27 17:48:04.068245 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.068233 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" containerName="test-curl-7411-139766023919200" Jul 27 17:48:04.068293 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.068247 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" containerName="test-curl-7411-139766023919200" Jul 27 17:48:04.068343 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.068328 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" containerName="test-curl-7411-139766023919200" Jul 27 17:48:04.071429 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.071412 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.075457 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.075438 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-d401e5-openshift-default-dockercfg-65w47\"" Jul 27 17:48:04.091396 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-credential-socket\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-data\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091429 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9wfc\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-kube-api-access-b9wfc\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091446 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-envoy\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091503 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091500 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-certs\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091660 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091540 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-podinfo\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091660 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091599 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-socket\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091660 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istiod-ca-cert\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.091752 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.091673 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-token\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.098287 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.098259 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26"] Jul 27 17:48:04.163274 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:48:04.163177 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-139766023919200\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:04.193101 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-certs\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193101 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193098 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-podinfo\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193334 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193122 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-socket\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193334 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istiod-ca-cert\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193334 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193263 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-token\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193334 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-credential-socket\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193574 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-data\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193574 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9wfc\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-kube-api-access-b9wfc\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193574 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-envoy\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193574 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193555 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-certs\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193774 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193602 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-socket\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193833 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193810 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-credential-socket\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.193892 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.193857 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-data\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.194363 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.194339 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istiod-ca-cert\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.195722 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.195698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-envoy\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.195983 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.195964 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-podinfo\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.200010 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.199983 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" path="/var/lib/kubelet/pods/01c35c51-e858-4269-b4b2-e7303cdeeda2/volumes" Jul 27 17:48:04.208951 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.208923 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-token\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.212978 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:48:04.212949 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-139766023919200\" is forbidden: User \"system:node:ip-10-0-135-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-54.ec2.internal' and this object" podUID="01c35c51-e858-4269-b4b2-e7303cdeeda2" pod="opendatahub/test-curl-7411-139766023919200" Jul 27 17:48:04.214160 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.214135 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9wfc\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-kube-api-access-b9wfc\") pod \"e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.382439 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.382365 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:04.559717 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:48:04.559687 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bed7d0f_6619_4f19_afc5_3aef30dd1753.slice/crio-d53b6972b709e3f90b797fc4c8acd1e953bcf2bd92b30195844515fe9cbc8223 WatchSource:0}: Error finding container d53b6972b709e3f90b797fc4c8acd1e953bcf2bd92b30195844515fe9cbc8223: Status 404 returned error can't find the container with id d53b6972b709e3f90b797fc4c8acd1e953bcf2bd92b30195844515fe9cbc8223 Jul 27 17:48:04.560333 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.560310 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26"] Jul 27 17:48:04.561894 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.561863 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:48:04.561978 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.561925 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:48:04.561978 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.561954 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:48:04.866664 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.866631 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" event={"ID":"9bed7d0f-6619-4f19-afc5-3aef30dd1753","Type":"ContainerStarted","Data":"7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce"} Jul 27 17:48:04.866860 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.866691 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" event={"ID":"9bed7d0f-6619-4f19-afc5-3aef30dd1753","Type":"ContainerStarted","Data":"d53b6972b709e3f90b797fc4c8acd1e953bcf2bd92b30195844515fe9cbc8223"} Jul 27 17:48:04.895169 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:04.895114 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" podStartSLOduration=0.895096847 podStartE2EDuration="895.096847ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:48:04.894550868 +0000 UTC m=+3091.178710151" watchObservedRunningTime="2026-07-27 17:48:04.895096847 +0000 UTC m=+3091.179256129" Jul 27 17:48:05.383598 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:05.383558 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:05.388956 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:05.388928 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:05.872463 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:05.872423 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:05.873505 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:05.873485 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:24.991665 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:24.991627 2582 generic.go:356] "Generic (PLEG): container finished" podID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerID="7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce" exitCode=0 Jul 27 17:48:24.992199 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:24.991692 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" event={"ID":"9bed7d0f-6619-4f19-afc5-3aef30dd1753","Type":"ContainerDied","Data":"7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce"} Jul 27 17:48:24.992199 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:24.992118 2582 scope.go:117] "RemoveContainer" containerID="7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce" Jul 27 17:48:24.993256 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:24.993226 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:48:24.993323 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:24.993308 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:48:24.993360 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:24.993344 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:48:26.002298 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:26.002256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" event={"ID":"9bed7d0f-6619-4f19-afc5-3aef30dd1753","Type":"ContainerStarted","Data":"fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075"} Jul 27 17:48:26.383555 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:26.383520 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:26.383849 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:26.383828 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 27 17:48:26.383901 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:26.383878 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 27 17:48:27.382801 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:27.382767 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 27 17:48:27.383217 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:27.382838 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 27 17:48:28.383767 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:28.383730 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 27 17:48:28.384258 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:28.383798 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 27 17:48:29.383343 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:29.383306 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 27 17:48:29.383542 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:29.383358 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 27 17:48:30.386656 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:30.386624 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:30.387035 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:30.386885 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:48:30.387614 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:48:30.387598 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:49:21.359640 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.359609 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nvs7g"] Jul 27 17:49:21.363019 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.362999 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:21.373150 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.373080 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nvs7g"] Jul 27 17:49:21.477439 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.477401 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dwmp\" (UniqueName: \"kubernetes.io/projected/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f-kube-api-access-7dwmp\") pod \"kuadrant-operator-catalog-nvs7g\" (UID: \"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f\") " pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:21.578730 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.578694 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwmp\" (UniqueName: \"kubernetes.io/projected/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f-kube-api-access-7dwmp\") pod \"kuadrant-operator-catalog-nvs7g\" (UID: \"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f\") " pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:21.588855 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.588825 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dwmp\" (UniqueName: \"kubernetes.io/projected/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f-kube-api-access-7dwmp\") pod \"kuadrant-operator-catalog-nvs7g\" (UID: \"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f\") " pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:21.673689 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.673588 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:21.816214 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:21.816186 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nvs7g"] Jul 27 17:49:21.818779 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:49:21.818753 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ea7b8a_ca0d_44cb_8515_7c2a42effe1f.slice/crio-559b80500ad3ffaa3fa294baa36218e5710ed5a27f660448e958766626e20d69 WatchSource:0}: Error finding container 559b80500ad3ffaa3fa294baa36218e5710ed5a27f660448e958766626e20d69: Status 404 returned error can't find the container with id 559b80500ad3ffaa3fa294baa36218e5710ed5a27f660448e958766626e20d69 Jul 27 17:49:22.352629 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:22.352593 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" event={"ID":"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f","Type":"ContainerStarted","Data":"559b80500ad3ffaa3fa294baa36218e5710ed5a27f660448e958766626e20d69"} Jul 27 17:49:23.359788 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:23.359751 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" event={"ID":"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f","Type":"ContainerStarted","Data":"01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc"} Jul 27 17:49:23.381133 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:23.381080 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" podStartSLOduration=1.809482434 podStartE2EDuration="2.381064242s" totalImagesPullingTime="571.581808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:49:23.377745471 +0000 UTC m=+3169.661904754" watchObservedRunningTime="2026-07-27 17:49:23.381064242 +0000 UTC m=+3169.665223535" Jul 27 17:49:31.674143 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:31.674096 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:31.674818 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:31.674184 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:31.696142 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:31.696114 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:32.436186 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:32.436158 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:32.490863 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:32.490823 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nvs7g"] Jul 27 17:49:34.427230 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:34.427159 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" podUID="58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" containerName="registry-server" containerID="cri-o://01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc" gracePeriod=2 Jul 27 17:49:34.679265 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:34.679203 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:34.812275 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:34.812237 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dwmp\" (UniqueName: \"kubernetes.io/projected/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f-kube-api-access-7dwmp\") pod \"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f\" (UID: \"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f\") " Jul 27 17:49:34.814282 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:34.814253 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f-kube-api-access-7dwmp" (OuterVolumeSpecName: "kube-api-access-7dwmp") pod "58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" (UID: "58ea7b8a-ca0d-44cb-8515-7c2a42effe1f"). InnerVolumeSpecName "kube-api-access-7dwmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:49:34.913745 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:34.913705 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dwmp\" (UniqueName: \"kubernetes.io/projected/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f-kube-api-access-7dwmp\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:49:35.434535 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.434495 2582 generic.go:356] "Generic (PLEG): container finished" podID="58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" containerID="01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc" exitCode=0 Jul 27 17:49:35.434923 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.434549 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" Jul 27 17:49:35.434923 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.434580 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" event={"ID":"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f","Type":"ContainerDied","Data":"01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc"} Jul 27 17:49:35.434923 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.434643 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nvs7g" event={"ID":"58ea7b8a-ca0d-44cb-8515-7c2a42effe1f","Type":"ContainerDied","Data":"559b80500ad3ffaa3fa294baa36218e5710ed5a27f660448e958766626e20d69"} Jul 27 17:49:35.434923 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.434666 2582 scope.go:117] "RemoveContainer" containerID="01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc" Jul 27 17:49:35.445013 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.444995 2582 scope.go:117] "RemoveContainer" containerID="01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc" Jul 27 17:49:35.445248 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:49:35.445231 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc\": container with ID starting with 01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc not found: ID does not exist" containerID="01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc" Jul 27 17:49:35.445290 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.445255 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc"} err="failed to get container status \"01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc\": rpc error: code = NotFound desc = could not find container \"01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc\": container with ID starting with 01eb0e156a73fd3db317891e2ced59c98ec67ef090c1c4d8e83ce92f67baa4bc not found: ID does not exist" Jul 27 17:49:35.459716 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.459687 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nvs7g"] Jul 27 17:49:35.465734 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:35.465712 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nvs7g"] Jul 27 17:49:36.199328 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:36.199294 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" path="/var/lib/kubelet/pods/58ea7b8a-ca0d-44cb-8515-7c2a42effe1f/volumes" Jul 27 17:49:40.428533 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.428502 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk"] Jul 27 17:49:40.428929 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.428916 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" containerName="registry-server" Jul 27 17:49:40.428975 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.428931 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" containerName="registry-server" Jul 27 17:49:40.429014 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.429008 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="58ea7b8a-ca0d-44cb-8515-7c2a42effe1f" containerName="registry-server" Jul 27 17:49:40.433370 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.433351 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.436608 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.436588 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-4df9d8-openshift-default-dockercfg-njtrc\"" Jul 27 17:49:40.446147 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.446122 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk"] Jul 27 17:49:40.571873 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.571828 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-socket\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572061 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.571886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-token\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572061 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.571924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-certs\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572061 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.571955 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-envoy\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572061 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.571998 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-podinfo\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572209 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.572096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-credential-socket\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572209 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.572127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-data\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572209 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.572148 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.572209 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.572170 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-kube-api-access-np2s5\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673533 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673507 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-credential-socket\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673629 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673545 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-data\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673629 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673569 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673629 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673585 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-kube-api-access-np2s5\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673629 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673620 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-socket\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673828 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673678 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-token\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673828 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-certs\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673828 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673762 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-envoy\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.673828 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-podinfo\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.674042 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.673977 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-data\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.674129 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.674108 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-credential-socket\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.674190 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.674111 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-socket\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.674190 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.674159 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-certs\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.674475 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.674454 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.676042 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.676021 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-envoy\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.676257 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.676239 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-podinfo\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.682588 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.682527 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-token\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.682695 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.682629 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-kube-api-access-np2s5\") pod \"e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.745120 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.745091 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:40.898439 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.898402 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk"] Jul 27 17:49:40.899656 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:49:40.899627 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e7eb4d_7c94_414a_8147_ba724d9a851f.slice/crio-8e7ede61f4c974da72ef9ad79d1bdd6fafa2001475984363eb9965b15662e0cb WatchSource:0}: Error finding container 8e7ede61f4c974da72ef9ad79d1bdd6fafa2001475984363eb9965b15662e0cb: Status 404 returned error can't find the container with id 8e7ede61f4c974da72ef9ad79d1bdd6fafa2001475984363eb9965b15662e0cb Jul 27 17:49:40.902048 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.902015 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:49:40.902140 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.902083 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:49:40.902140 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:40.902112 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:49:41.474591 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:41.474557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" event={"ID":"c1e7eb4d-7c94-414a-8147-ba724d9a851f","Type":"ContainerStarted","Data":"af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc"} Jul 27 17:49:41.475051 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:41.474616 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" event={"ID":"c1e7eb4d-7c94-414a-8147-ba724d9a851f","Type":"ContainerStarted","Data":"8e7ede61f4c974da72ef9ad79d1bdd6fafa2001475984363eb9965b15662e0cb"} Jul 27 17:49:41.504516 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:41.504462 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podStartSLOduration=1.504444627 podStartE2EDuration="1.504444627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:49:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:49:41.503428438 +0000 UTC m=+3187.787587724" watchObservedRunningTime="2026-07-27 17:49:41.504444627 +0000 UTC m=+3187.788603909" Jul 27 17:49:41.745432 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:41.745314 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:41.750740 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:41.750707 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:42.480482 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:42.480451 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:42.481645 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:42.481627 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:49:43.565191 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.565160 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4"] Jul 27 17:49:43.569055 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.569031 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.572088 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.572065 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4df9d8-serving-cert\"" Jul 27 17:49:43.582483 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.582455 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4"] Jul 27 17:49:43.703703 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.703661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfwfw\" (UniqueName: \"kubernetes.io/projected/336d4530-6336-4423-aabb-934e40517689-kube-api-access-sfwfw\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.703703 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.703707 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.703929 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.703785 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336d4530-6336-4423-aabb-934e40517689-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.805315 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.805280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfwfw\" (UniqueName: \"kubernetes.io/projected/336d4530-6336-4423-aabb-934e40517689-kube-api-access-sfwfw\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.805315 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.805316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.805589 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.805346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336d4530-6336-4423-aabb-934e40517689-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.805675 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:49:43.805630 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-serving-cert: secret "maas-api-e2e-ipp-cleanup-4df9d8-serving-cert" not found Jul 27 17:49:43.805957 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:49:43.805930 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls podName:336d4530-6336-4423-aabb-934e40517689 nodeName:}" failed. No retries permitted until 2026-07-27 17:49:44.305854815 +0000 UTC m=+3190.590014093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" (UID: "336d4530-6336-4423-aabb-934e40517689") : secret "maas-api-e2e-ipp-cleanup-4df9d8-serving-cert" not found Jul 27 17:49:43.806165 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.806139 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336d4530-6336-4423-aabb-934e40517689-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:43.844487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:43.844450 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfwfw\" (UniqueName: \"kubernetes.io/projected/336d4530-6336-4423-aabb-934e40517689-kube-api-access-sfwfw\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:44.311765 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:44.311686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:44.314012 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:44.313986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:44.481313 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:44.481281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:44.876995 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:44.876969 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4"] Jul 27 17:49:44.878674 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:49:44.878649 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336d4530_6336_4423_aabb_934e40517689.slice/crio-fa659331b4e288e1b42c921f1fe9920b0e0ae67bb2ab4f5cd35d821e9ebc4f82 WatchSource:0}: Error finding container fa659331b4e288e1b42c921f1fe9920b0e0ae67bb2ab4f5cd35d821e9ebc4f82: Status 404 returned error can't find the container with id fa659331b4e288e1b42c921f1fe9920b0e0ae67bb2ab4f5cd35d821e9ebc4f82 Jul 27 17:49:45.502875 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:45.502833 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" event={"ID":"336d4530-6336-4423-aabb-934e40517689","Type":"ContainerStarted","Data":"fa659331b4e288e1b42c921f1fe9920b0e0ae67bb2ab4f5cd35d821e9ebc4f82"} Jul 27 17:49:46.510651 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:46.510543 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" event={"ID":"336d4530-6336-4423-aabb-934e40517689","Type":"ContainerStarted","Data":"339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5"} Jul 27 17:49:46.510651 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:46.510622 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:52.522629 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:52.522599 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:49:52.548038 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:52.547983 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" podStartSLOduration=8.286941374 podStartE2EDuration="9.547965496s" totalImagesPullingTime="1.261024122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:49:46.532886218 +0000 UTC m=+3192.817045500" watchObservedRunningTime="2026-07-27 17:49:52.547965496 +0000 UTC m=+3198.832124779" Jul 27 17:49:58.950981 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:58.950940 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr"] Jul 27 17:49:58.955812 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:58.955789 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:49:58.959197 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:58.959158 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:49:58.964260 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:58.964236 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr"] Jul 27 17:49:59.065837 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:59.065789 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7111b-8bc1-4629-8e2f-446ede40ccc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr\" (UID: \"ffb7111b-8bc1-4629-8e2f-446ede40ccc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:49:59.166816 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:59.166776 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7111b-8bc1-4629-8e2f-446ede40ccc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr\" (UID: \"ffb7111b-8bc1-4629-8e2f-446ede40ccc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:49:59.167445 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:59.167425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7111b-8bc1-4629-8e2f-446ede40ccc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr\" (UID: \"ffb7111b-8bc1-4629-8e2f-446ede40ccc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:49:59.269632 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:59.269560 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:49:59.428839 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:59.428798 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr"] Jul 27 17:49:59.430026 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:49:59.430002 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb7111b_8bc1_4629_8e2f_446ede40ccc1.slice/crio-a6f35c5c9619b477cf4807db7216f2ef3f65b508154fa7e7d5a4cef37b0ff70f WatchSource:0}: Error finding container a6f35c5c9619b477cf4807db7216f2ef3f65b508154fa7e7d5a4cef37b0ff70f: Status 404 returned error can't find the container with id a6f35c5c9619b477cf4807db7216f2ef3f65b508154fa7e7d5a4cef37b0ff70f Jul 27 17:49:59.594094 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:49:59.594011 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" event={"ID":"ffb7111b-8bc1-4629-8e2f-446ede40ccc1","Type":"ContainerStarted","Data":"a6f35c5c9619b477cf4807db7216f2ef3f65b508154fa7e7d5a4cef37b0ff70f"} Jul 27 17:50:00.603361 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:00.603325 2582 generic.go:356] "Generic (PLEG): container finished" podID="ffb7111b-8bc1-4629-8e2f-446ede40ccc1" containerID="4dafc97173c5140a818297da86835382a7d9b63bdf7900ac91b480c1e798c906" exitCode=0 Jul 27 17:50:00.603765 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:00.603414 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" event={"ID":"ffb7111b-8bc1-4629-8e2f-446ede40ccc1","Type":"ContainerDied","Data":"4dafc97173c5140a818297da86835382a7d9b63bdf7900ac91b480c1e798c906"} Jul 27 17:50:01.611167 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.611127 2582 generic.go:356] "Generic (PLEG): container finished" podID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerID="af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc" exitCode=0 Jul 27 17:50:01.611605 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.611201 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" event={"ID":"c1e7eb4d-7c94-414a-8147-ba724d9a851f","Type":"ContainerDied","Data":"af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc"} Jul 27 17:50:01.611656 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.611620 2582 scope.go:117] "RemoveContainer" containerID="af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc" Jul 27 17:50:01.612872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.612839 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:50:01.612979 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.612919 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:50:01.612979 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.612956 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 17:50:01.763030 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.763000 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:50:01.792517 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.792481 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7111b-8bc1-4629-8e2f-446ede40ccc1-maas-api-service-ca\") pod \"ffb7111b-8bc1-4629-8e2f-446ede40ccc1\" (UID: \"ffb7111b-8bc1-4629-8e2f-446ede40ccc1\") " Jul 27 17:50:01.792882 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.792853 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb7111b-8bc1-4629-8e2f-446ede40ccc1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ffb7111b-8bc1-4629-8e2f-446ede40ccc1" (UID: "ffb7111b-8bc1-4629-8e2f-446ede40ccc1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:50:01.792996 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:01.792965 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffb7111b-8bc1-4629-8e2f-446ede40ccc1-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:02.624671 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.623854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" event={"ID":"c1e7eb4d-7c94-414a-8147-ba724d9a851f","Type":"ContainerStarted","Data":"edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c"} Jul 27 17:50:02.627338 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.627299 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" event={"ID":"ffb7111b-8bc1-4629-8e2f-446ede40ccc1","Type":"ContainerDied","Data":"a6f35c5c9619b477cf4807db7216f2ef3f65b508154fa7e7d5a4cef37b0ff70f"} Jul 27 17:50:02.627531 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.627362 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6f35c5c9619b477cf4807db7216f2ef3f65b508154fa7e7d5a4cef37b0ff70f" Jul 27 17:50:02.627531 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.627451 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr" Jul 27 17:50:02.745936 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.745897 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:50:02.746173 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.746155 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 27 17:50:02.746228 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:02.746209 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 27 17:50:03.745649 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:03.745608 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 27 17:50:03.746035 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:03.745667 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 27 17:50:04.745642 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:04.745607 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 27 17:50:04.745826 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:04.745665 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 27 17:50:05.746139 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:05.746096 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 27 17:50:05.746559 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:05.746157 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 27 17:50:06.749303 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:06.749270 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:50:06.749781 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:06.749706 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:50:06.750417 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:06.750371 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:50:09.634445 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.634406 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4"] Jul 27 17:50:09.634936 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.634625 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" podUID="336d4530-6336-4423-aabb-934e40517689" containerName="maas-api" containerID="cri-o://339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5" gracePeriod=30 Jul 27 17:50:09.897121 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.897054 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:50:09.978829 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.978798 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls\") pod \"336d4530-6336-4423-aabb-934e40517689\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " Jul 27 17:50:09.978998 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.978854 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfwfw\" (UniqueName: \"kubernetes.io/projected/336d4530-6336-4423-aabb-934e40517689-kube-api-access-sfwfw\") pod \"336d4530-6336-4423-aabb-934e40517689\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " Jul 27 17:50:09.978998 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.978898 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336d4530-6336-4423-aabb-934e40517689-openshift-service-ca\") pod \"336d4530-6336-4423-aabb-934e40517689\" (UID: \"336d4530-6336-4423-aabb-934e40517689\") " Jul 27 17:50:09.979304 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.979272 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/336d4530-6336-4423-aabb-934e40517689-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "336d4530-6336-4423-aabb-934e40517689" (UID: "336d4530-6336-4423-aabb-934e40517689"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:50:09.981231 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.981197 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336d4530-6336-4423-aabb-934e40517689-kube-api-access-sfwfw" (OuterVolumeSpecName: "kube-api-access-sfwfw") pod "336d4530-6336-4423-aabb-934e40517689" (UID: "336d4530-6336-4423-aabb-934e40517689"). InnerVolumeSpecName "kube-api-access-sfwfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:50:09.981231 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:09.981210 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "336d4530-6336-4423-aabb-934e40517689" (UID: "336d4530-6336-4423-aabb-934e40517689"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 17:50:10.080213 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.080176 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336d4530-6336-4423-aabb-934e40517689-maas-api-tls\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:10.080213 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.080208 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfwfw\" (UniqueName: \"kubernetes.io/projected/336d4530-6336-4423-aabb-934e40517689-kube-api-access-sfwfw\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:10.080213 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.080217 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336d4530-6336-4423-aabb-934e40517689-openshift-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:10.680555 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.680522 2582 generic.go:356] "Generic (PLEG): container finished" podID="336d4530-6336-4423-aabb-934e40517689" containerID="339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5" exitCode=0 Jul 27 17:50:10.680555 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.680565 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" Jul 27 17:50:10.681106 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.680600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" event={"ID":"336d4530-6336-4423-aabb-934e40517689","Type":"ContainerDied","Data":"339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5"} Jul 27 17:50:10.681106 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.680658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4" event={"ID":"336d4530-6336-4423-aabb-934e40517689","Type":"ContainerDied","Data":"fa659331b4e288e1b42c921f1fe9920b0e0ae67bb2ab4f5cd35d821e9ebc4f82"} Jul 27 17:50:10.681106 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.680680 2582 scope.go:117] "RemoveContainer" containerID="339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5" Jul 27 17:50:10.690053 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.690033 2582 scope.go:117] "RemoveContainer" containerID="339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5" Jul 27 17:50:10.690352 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:50:10.690331 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5\": container with ID starting with 339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5 not found: ID does not exist" containerID="339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5" Jul 27 17:50:10.690456 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.690362 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5"} err="failed to get container status \"339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5\": rpc error: code = NotFound desc = could not find container \"339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5\": container with ID starting with 339dee77dd29ce38fc6281381aba34e1cae7de778f5e16a328d1a12ce31f99b5 not found: ID does not exist" Jul 27 17:50:10.701272 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.701250 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4"] Jul 27 17:50:10.704973 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:10.704951 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4df9d8-65bb7dfc86-scbc4"] Jul 27 17:50:12.199701 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:12.199667 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="336d4530-6336-4423-aabb-934e40517689" path="/var/lib/kubelet/pods/336d4530-6336-4423-aabb-934e40517689/volumes" Jul 27 17:50:14.062764 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:14.062729 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr"] Jul 27 17:50:14.065469 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:14.065448 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-4df9d8-d9wnr"] Jul 27 17:50:14.198949 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:14.198916 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffb7111b-8bc1-4629-8e2f-446ede40ccc1" path="/var/lib/kubelet/pods/ffb7111b-8bc1-4629-8e2f-446ede40ccc1/volumes" Jul 27 17:50:20.693366 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:20.693328 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk"] Jul 27 17:50:20.693847 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:20.693602 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" containerID="cri-o://edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c" gracePeriod=30 Jul 27 17:50:21.752673 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:21.752634 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 27 17:50:21.753064 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:21.752719 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 17:50:24.622766 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.622730 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk"] Jul 27 17:50:24.623168 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.623156 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb7111b-8bc1-4629-8e2f-446ede40ccc1" containerName="revoke-keys" Jul 27 17:50:24.623214 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.623170 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb7111b-8bc1-4629-8e2f-446ede40ccc1" containerName="revoke-keys" Jul 27 17:50:24.623214 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.623182 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="336d4530-6336-4423-aabb-934e40517689" containerName="maas-api" Jul 27 17:50:24.623214 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.623188 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="336d4530-6336-4423-aabb-934e40517689" containerName="maas-api" Jul 27 17:50:24.623305 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.623267 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="336d4530-6336-4423-aabb-934e40517689" containerName="maas-api" Jul 27 17:50:24.623305 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.623278 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb7111b-8bc1-4629-8e2f-446ede40ccc1" containerName="revoke-keys" Jul 27 17:50:24.626502 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.626486 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:24.629672 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.629646 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:50:24.636180 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.636154 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk"] Jul 27 17:50:24.732010 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.731979 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85317031-019b-4a39-b201-da774657e7fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk\" (UID: \"85317031-019b-4a39-b201-da774657e7fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:24.833248 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.833207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85317031-019b-4a39-b201-da774657e7fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk\" (UID: \"85317031-019b-4a39-b201-da774657e7fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:24.834468 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.834438 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85317031-019b-4a39-b201-da774657e7fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk\" (UID: \"85317031-019b-4a39-b201-da774657e7fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:24.939329 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:24.939237 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:25.291694 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:25.291668 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk"] Jul 27 17:50:25.292613 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:50:25.292579 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85317031_019b_4a39_b201_da774657e7fd.slice/crio-fc48c26da6f4047680de0642be467df171dbe1df52d244c8fe67baff70d5b06e WatchSource:0}: Error finding container fc48c26da6f4047680de0642be467df171dbe1df52d244c8fe67baff70d5b06e: Status 404 returned error can't find the container with id fc48c26da6f4047680de0642be467df171dbe1df52d244c8fe67baff70d5b06e Jul 27 17:50:25.783292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:25.783252 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" event={"ID":"85317031-019b-4a39-b201-da774657e7fd","Type":"ContainerStarted","Data":"ad611b5436eeaba4f511f7800ef7d0f93d5ca6a42b7013a5468bb22b5bb8fa17"} Jul 27 17:50:25.783705 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:25.783310 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" event={"ID":"85317031-019b-4a39-b201-da774657e7fd","Type":"ContainerStarted","Data":"fc48c26da6f4047680de0642be467df171dbe1df52d244c8fe67baff70d5b06e"} Jul 27 17:50:25.806125 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:25.806073 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" podStartSLOduration=1.80605814 podStartE2EDuration="1.80605814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:50:25.803483408 +0000 UTC m=+3232.087642688" watchObservedRunningTime="2026-07-27 17:50:25.80605814 +0000 UTC m=+3232.090217426" Jul 27 17:50:25.938805 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:25.938780 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:50:26.047876 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.047838 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-certs\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048067 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.047903 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-kube-api-access-np2s5\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048067 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.047936 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-credential-socket\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048067 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.047977 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-envoy\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048067 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048021 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-data\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048068 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istiod-ca-cert\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048123 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-socket\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048121 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:50:26.048292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048229 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-token\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048263 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:50:26.048292 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048292 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-podinfo\") pod \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\" (UID: \"c1e7eb4d-7c94-414a-8147-ba724d9a851f\") " Jul 27 17:50:26.048652 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048541 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:50:26.048652 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048584 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:50:26.048763 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048732 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istiod-ca-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.048763 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048752 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.048872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048765 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-workload-certs\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.048872 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.048781 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-credential-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.049396 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.049355 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:50:26.050456 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.050436 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:50:26.050669 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.050646 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:50:26.050669 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.050662 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-kube-api-access-np2s5" (OuterVolumeSpecName: "kube-api-access-np2s5") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "kube-api-access-np2s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:50:26.050803 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.050746 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1e7eb4d-7c94-414a-8147-ba724d9a851f" (UID: "c1e7eb4d-7c94-414a-8147-ba724d9a851f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 17:50:26.149906 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.149833 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-token\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.149906 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.149864 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-podinfo\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.149906 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.149873 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np2s5\" (UniqueName: \"kubernetes.io/projected/c1e7eb4d-7c94-414a-8147-ba724d9a851f-kube-api-access-np2s5\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.149906 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.149883 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-envoy\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.149906 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.149893 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1e7eb4d-7c94-414a-8147-ba724d9a851f-istio-data\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:26.791294 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.791253 2582 generic.go:356] "Generic (PLEG): container finished" podID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerID="edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c" exitCode=0 Jul 27 17:50:26.791759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.791306 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" Jul 27 17:50:26.791759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.791337 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" event={"ID":"c1e7eb4d-7c94-414a-8147-ba724d9a851f","Type":"ContainerDied","Data":"edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c"} Jul 27 17:50:26.791759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.791409 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk" event={"ID":"c1e7eb4d-7c94-414a-8147-ba724d9a851f","Type":"ContainerDied","Data":"8e7ede61f4c974da72ef9ad79d1bdd6fafa2001475984363eb9965b15662e0cb"} Jul 27 17:50:26.791759 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.791435 2582 scope.go:117] "RemoveContainer" containerID="edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c" Jul 27 17:50:26.792971 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.792948 2582 generic.go:356] "Generic (PLEG): container finished" podID="85317031-019b-4a39-b201-da774657e7fd" containerID="ad611b5436eeaba4f511f7800ef7d0f93d5ca6a42b7013a5468bb22b5bb8fa17" exitCode=0 Jul 27 17:50:26.793084 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.793017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" event={"ID":"85317031-019b-4a39-b201-da774657e7fd","Type":"ContainerDied","Data":"ad611b5436eeaba4f511f7800ef7d0f93d5ca6a42b7013a5468bb22b5bb8fa17"} Jul 27 17:50:26.803327 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.803309 2582 scope.go:117] "RemoveContainer" containerID="af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc" Jul 27 17:50:26.811877 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.811862 2582 scope.go:117] "RemoveContainer" containerID="edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c" Jul 27 17:50:26.812148 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:50:26.812125 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c\": container with ID starting with edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c not found: ID does not exist" containerID="edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c" Jul 27 17:50:26.812234 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.812155 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c"} err="failed to get container status \"edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c\": rpc error: code = NotFound desc = could not find container \"edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c\": container with ID starting with edb829deb20248e4b0499d8995d135c895862532a48b12facbed9e5f3e3ea38c not found: ID does not exist" Jul 27 17:50:26.812234 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.812174 2582 scope.go:117] "RemoveContainer" containerID="af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc" Jul 27 17:50:26.812478 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:50:26.812369 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc\": container with ID starting with af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc not found: ID does not exist" containerID="af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc" Jul 27 17:50:26.812525 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.812482 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc"} err="failed to get container status \"af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc\": rpc error: code = NotFound desc = could not find container \"af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc\": container with ID starting with af9622755d1df051c999d931b2ebcb9ce7205984575251d563a74e7295b99afc not found: ID does not exist" Jul 27 17:50:26.846314 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.846286 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk"] Jul 27 17:50:26.852839 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:26.852814 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df9d8-openshift-default-6f4c497f98-nvtqk"] Jul 27 17:50:27.940139 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:27.940114 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:28.069517 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.069435 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85317031-019b-4a39-b201-da774657e7fd-maas-api-service-ca\") pod \"85317031-019b-4a39-b201-da774657e7fd\" (UID: \"85317031-019b-4a39-b201-da774657e7fd\") " Jul 27 17:50:28.069780 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.069759 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85317031-019b-4a39-b201-da774657e7fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85317031-019b-4a39-b201-da774657e7fd" (UID: "85317031-019b-4a39-b201-da774657e7fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:50:28.170516 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.170474 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85317031-019b-4a39-b201-da774657e7fd-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:50:28.200062 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.200030 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" path="/var/lib/kubelet/pods/c1e7eb4d-7c94-414a-8147-ba724d9a851f/volumes" Jul 27 17:50:28.808089 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.808054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" event={"ID":"85317031-019b-4a39-b201-da774657e7fd","Type":"ContainerDied","Data":"fc48c26da6f4047680de0642be467df171dbe1df52d244c8fe67baff70d5b06e"} Jul 27 17:50:28.808271 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.808108 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc48c26da6f4047680de0642be467df171dbe1df52d244c8fe67baff70d5b06e" Jul 27 17:50:28.808271 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:28.808066 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk" Jul 27 17:50:44.805249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:44.805209 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk"] Jul 27 17:50:44.813562 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:44.813534 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d401e5-tmtdk"] Jul 27 17:50:46.198550 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:50:46.198515 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85317031-019b-4a39-b201-da774657e7fd" path="/var/lib/kubelet/pods/85317031-019b-4a39-b201-da774657e7fd/volumes" Jul 27 17:51:39.891673 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:39.891634 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26"] Jul 27 17:51:39.892271 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:39.891828 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" containerID="cri-o://fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075" gracePeriod=30 Jul 27 17:51:40.264986 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.264901 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn"] Jul 27 17:51:40.265321 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265308 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" Jul 27 17:51:40.265404 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265322 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" Jul 27 17:51:40.265404 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265332 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" Jul 27 17:51:40.265404 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265338 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" Jul 27 17:51:40.265404 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265361 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85317031-019b-4a39-b201-da774657e7fd" containerName="revoke-keys" Jul 27 17:51:40.265404 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265366 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="85317031-019b-4a39-b201-da774657e7fd" containerName="revoke-keys" Jul 27 17:51:40.265565 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265467 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="85317031-019b-4a39-b201-da774657e7fd" containerName="revoke-keys" Jul 27 17:51:40.265565 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.265478 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" Jul 27 17:51:40.268748 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.268730 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:40.274758 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.274735 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:51:40.278365 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.278337 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn"] Jul 27 17:51:40.390937 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.390897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94fbcaa-5acb-4042-8e49-1a1cd1762f21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn\" (UID: \"a94fbcaa-5acb-4042-8e49-1a1cd1762f21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:40.491924 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.491887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94fbcaa-5acb-4042-8e49-1a1cd1762f21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn\" (UID: \"a94fbcaa-5acb-4042-8e49-1a1cd1762f21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:40.492617 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.492595 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94fbcaa-5acb-4042-8e49-1a1cd1762f21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn\" (UID: \"a94fbcaa-5acb-4042-8e49-1a1cd1762f21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:40.580953 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.580844 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:40.724031 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:40.723989 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn"] Jul 27 17:51:40.724163 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:51:40.724145 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94fbcaa_5acb_4042_8e49_1a1cd1762f21.slice/crio-e8ed42c3a1aba50d7063d356e48595a6a78d0830eac724eb157e3220a65a42ad WatchSource:0}: Error finding container e8ed42c3a1aba50d7063d356e48595a6a78d0830eac724eb157e3220a65a42ad: Status 404 returned error can't find the container with id e8ed42c3a1aba50d7063d356e48595a6a78d0830eac724eb157e3220a65a42ad Jul 27 17:51:41.244804 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:41.244718 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" event={"ID":"a94fbcaa-5acb-4042-8e49-1a1cd1762f21","Type":"ContainerStarted","Data":"527f388e0d43fa0a1684f828611262b47aada36e40ad92b9b2770b0d6cf29c8f"} Jul 27 17:51:41.244804 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:41.244781 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" event={"ID":"a94fbcaa-5acb-4042-8e49-1a1cd1762f21","Type":"ContainerStarted","Data":"e8ed42c3a1aba50d7063d356e48595a6a78d0830eac724eb157e3220a65a42ad"} Jul 27 17:51:41.264049 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:41.263998 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" podStartSLOduration=1.263983862 podStartE2EDuration="1.263983862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:51:41.261779564 +0000 UTC m=+3307.545938847" watchObservedRunningTime="2026-07-27 17:51:41.263983862 +0000 UTC m=+3307.548143145" Jul 27 17:51:42.252705 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:42.252666 2582 generic.go:356] "Generic (PLEG): container finished" podID="a94fbcaa-5acb-4042-8e49-1a1cd1762f21" containerID="527f388e0d43fa0a1684f828611262b47aada36e40ad92b9b2770b0d6cf29c8f" exitCode=0 Jul 27 17:51:42.253114 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:42.252754 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" event={"ID":"a94fbcaa-5acb-4042-8e49-1a1cd1762f21","Type":"ContainerDied","Data":"527f388e0d43fa0a1684f828611262b47aada36e40ad92b9b2770b0d6cf29c8f"} Jul 27 17:51:43.395100 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:43.395073 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:43.521161 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:43.521058 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94fbcaa-5acb-4042-8e49-1a1cd1762f21-maas-api-service-ca\") pod \"a94fbcaa-5acb-4042-8e49-1a1cd1762f21\" (UID: \"a94fbcaa-5acb-4042-8e49-1a1cd1762f21\") " Jul 27 17:51:43.521551 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:43.521521 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a94fbcaa-5acb-4042-8e49-1a1cd1762f21-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a94fbcaa-5acb-4042-8e49-1a1cd1762f21" (UID: "a94fbcaa-5acb-4042-8e49-1a1cd1762f21"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:51:43.622036 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:43.622000 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a94fbcaa-5acb-4042-8e49-1a1cd1762f21-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:44.266487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:44.266446 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" event={"ID":"a94fbcaa-5acb-4042-8e49-1a1cd1762f21","Type":"ContainerDied","Data":"e8ed42c3a1aba50d7063d356e48595a6a78d0830eac724eb157e3220a65a42ad"} Jul 27 17:51:44.266487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:44.266469 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn" Jul 27 17:51:44.266708 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:44.266498 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8ed42c3a1aba50d7063d356e48595a6a78d0830eac724eb157e3220a65a42ad" Jul 27 17:51:45.147951 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.147925 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:51:45.239255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239164 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-credential-socket\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239208 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-token\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239230 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istiod-ca-cert\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239255 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239256 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-certs\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239630 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239278 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-data\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239630 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239300 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9wfc\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-kube-api-access-b9wfc\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239630 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239338 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-socket\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239630 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239360 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-podinfo\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239630 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239582 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:51:45.239878 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239677 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-envoy\") pod \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\" (UID: \"9bed7d0f-6619-4f19-afc5-3aef30dd1753\") " Jul 27 17:51:45.239878 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239695 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:51:45.239878 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.239738 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:51:45.240068 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.240039 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:51:45.240127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.240083 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.240127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.240100 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istiod-ca-cert\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.240127 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.240114 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-workload-certs\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.240293 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.240228 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:51:45.241625 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.241607 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 17:51:45.241693 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.241628 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:51:45.241928 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.241909 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-kube-api-access-b9wfc" (OuterVolumeSpecName: "kube-api-access-b9wfc") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "kube-api-access-b9wfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 17:51:45.242020 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.242007 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bed7d0f-6619-4f19-afc5-3aef30dd1753" (UID: "9bed7d0f-6619-4f19-afc5-3aef30dd1753"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 17:51:45.274212 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.274175 2582 generic.go:356] "Generic (PLEG): container finished" podID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerID="fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075" exitCode=0 Jul 27 17:51:45.274356 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.274232 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" Jul 27 17:51:45.274356 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.274258 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" event={"ID":"9bed7d0f-6619-4f19-afc5-3aef30dd1753","Type":"ContainerDied","Data":"fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075"} Jul 27 17:51:45.274356 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.274316 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26" event={"ID":"9bed7d0f-6619-4f19-afc5-3aef30dd1753","Type":"ContainerDied","Data":"d53b6972b709e3f90b797fc4c8acd1e953bcf2bd92b30195844515fe9cbc8223"} Jul 27 17:51:45.274356 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.274338 2582 scope.go:117] "RemoveContainer" containerID="fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075" Jul 27 17:51:45.292456 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.292433 2582 scope.go:117] "RemoveContainer" containerID="7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce" Jul 27 17:51:45.304426 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.304403 2582 scope.go:117] "RemoveContainer" containerID="fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075" Jul 27 17:51:45.304724 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:51:45.304695 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075\": container with ID starting with fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075 not found: ID does not exist" containerID="fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075" Jul 27 17:51:45.304795 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.304735 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075"} err="failed to get container status \"fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075\": rpc error: code = NotFound desc = could not find container \"fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075\": container with ID starting with fb16f6f5867fddcc51dd06e1b6c7b4e14c2fce522a8a0ac43bb29474d43c3075 not found: ID does not exist" Jul 27 17:51:45.304795 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.304753 2582 scope.go:117] "RemoveContainer" containerID="7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce" Jul 27 17:51:45.305027 ip-10-0-135-54 kubenswrapper[2582]: E0727 17:51:45.305009 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce\": container with ID starting with 7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce not found: ID does not exist" containerID="7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce" Jul 27 17:51:45.305090 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.305032 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce"} err="failed to get container status \"7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce\": rpc error: code = NotFound desc = could not find container \"7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce\": container with ID starting with 7dad5523078665e38e7d9127c711350818286ca114a64a6f2c1ecfba5fd864ce not found: ID does not exist" Jul 27 17:51:45.306229 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.306205 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26"] Jul 27 17:51:45.312200 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.312178 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-d401e5-openshift-default-6f774c7f56-bcr26"] Jul 27 17:51:45.340957 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.340932 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-data\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.340957 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.340957 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9wfc\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-kube-api-access-b9wfc\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.341134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.340974 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-podinfo\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.341134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.340986 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-envoy\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.341134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.340997 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bed7d0f-6619-4f19-afc5-3aef30dd1753-credential-socket\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:45.341134 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:45.341007 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bed7d0f-6619-4f19-afc5-3aef30dd1753-istio-token\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:51:46.199212 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:46.199176 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" path="/var/lib/kubelet/pods/9bed7d0f-6619-4f19-afc5-3aef30dd1753/volumes" Jul 27 17:51:55.370635 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:55.370587 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn"] Jul 27 17:51:55.373256 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:55.373227 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-3bb3ae-zdxhn"] Jul 27 17:51:56.198406 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:51:56.198308 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a94fbcaa-5acb-4042-8e49-1a1cd1762f21" path="/var/lib/kubelet/pods/a94fbcaa-5acb-4042-8e49-1a1cd1762f21/volumes" Jul 27 17:52:02.849721 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.849683 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q"] Jul 27 17:52:02.850130 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850107 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" Jul 27 17:52:02.850130 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850118 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" Jul 27 17:52:02.850130 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850129 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850134 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850145 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a94fbcaa-5acb-4042-8e49-1a1cd1762f21" containerName="revoke-keys" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850151 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a94fbcaa-5acb-4042-8e49-1a1cd1762f21" containerName="revoke-keys" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850211 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e7eb4d-7c94-414a-8147-ba724d9a851f" containerName="istio-proxy" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850217 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850227 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bed7d0f-6619-4f19-afc5-3aef30dd1753" containerName="istio-proxy" Jul 27 17:52:02.850235 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.850235 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a94fbcaa-5acb-4042-8e49-1a1cd1762f21" containerName="revoke-keys" Jul 27 17:52:02.853631 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.853614 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:02.856881 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.856862 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:52:02.863157 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.863118 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q"] Jul 27 17:52:02.913697 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:02.913662 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-004618-dbj2q\" (UID: \"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:03.014487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:03.014448 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-004618-dbj2q\" (UID: \"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:03.015237 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:03.015217 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-004618-dbj2q\" (UID: \"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:03.165579 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:03.165488 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:03.522288 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:03.522249 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q"] Jul 27 17:52:03.524664 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:52:03.524634 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941ed9e3_d5c5_4d6f_b51b_7369dbc243a9.slice/crio-045ac2e47a9a04e7813538a1b4c380064c3c12c3b6d1e3156bd38cf686bf23ea WatchSource:0}: Error finding container 045ac2e47a9a04e7813538a1b4c380064c3c12c3b6d1e3156bd38cf686bf23ea: Status 404 returned error can't find the container with id 045ac2e47a9a04e7813538a1b4c380064c3c12c3b6d1e3156bd38cf686bf23ea Jul 27 17:52:04.392729 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:04.392687 2582 generic.go:356] "Generic (PLEG): container finished" podID="941ed9e3-d5c5-4d6f-b51b-7369dbc243a9" containerID="d7c4456180a343efd95184e8692f134fd5f235fe460abb71b0d13500c07b37b9" exitCode=0 Jul 27 17:52:04.393135 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:04.392744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" event={"ID":"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9","Type":"ContainerDied","Data":"d7c4456180a343efd95184e8692f134fd5f235fe460abb71b0d13500c07b37b9"} Jul 27 17:52:04.393135 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:04.392812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" event={"ID":"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9","Type":"ContainerStarted","Data":"045ac2e47a9a04e7813538a1b4c380064c3c12c3b6d1e3156bd38cf686bf23ea"} Jul 27 17:52:05.540108 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:05.540080 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:05.638658 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:05.638605 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9-maas-api-service-ca\") pod \"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9\" (UID: \"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9\") " Jul 27 17:52:05.638986 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:05.638963 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "941ed9e3-d5c5-4d6f-b51b-7369dbc243a9" (UID: "941ed9e3-d5c5-4d6f-b51b-7369dbc243a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:52:05.740035 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:05.739923 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:52:06.406502 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:06.406477 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" Jul 27 17:52:06.406687 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:06.406475 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q" event={"ID":"941ed9e3-d5c5-4d6f-b51b-7369dbc243a9","Type":"ContainerDied","Data":"045ac2e47a9a04e7813538a1b4c380064c3c12c3b6d1e3156bd38cf686bf23ea"} Jul 27 17:52:06.406687 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:06.406596 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="045ac2e47a9a04e7813538a1b4c380064c3c12c3b6d1e3156bd38cf686bf23ea" Jul 27 17:52:13.624936 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:13.624893 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss"] Jul 27 17:52:13.631074 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:13.631045 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-004618-29752905-sv6ss"] Jul 27 17:52:14.198764 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:14.198726 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1d2b3a8-ee85-4412-8fa2-3127f228562f" path="/var/lib/kubelet/pods/f1d2b3a8-ee85-4412-8fa2-3127f228562f/volumes" Jul 27 17:52:17.961180 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:17.961139 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q"] Jul 27 17:52:17.965250 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:17.965224 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-004618-dbj2q"] Jul 27 17:52:18.198932 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:18.198897 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="941ed9e3-d5c5-4d6f-b51b-7369dbc243a9" path="/var/lib/kubelet/pods/941ed9e3-d5c5-4d6f-b51b-7369dbc243a9/volumes" Jul 27 17:52:25.536368 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.536333 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9"] Jul 27 17:52:25.536834 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.536820 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="941ed9e3-d5c5-4d6f-b51b-7369dbc243a9" containerName="revoke-keys" Jul 27 17:52:25.536885 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.536836 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="941ed9e3-d5c5-4d6f-b51b-7369dbc243a9" containerName="revoke-keys" Jul 27 17:52:25.536949 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.536938 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="941ed9e3-d5c5-4d6f-b51b-7369dbc243a9" containerName="revoke-keys" Jul 27 17:52:25.540239 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.540220 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:25.543487 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.543470 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grztm\"" Jul 27 17:52:25.549976 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.549952 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9"] Jul 27 17:52:25.727931 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.727891 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/886037f3-28bf-47d6-b057-1bf7cf5cd1b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9\" (UID: \"886037f3-28bf-47d6-b057-1bf7cf5cd1b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:25.829054 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.828959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/886037f3-28bf-47d6-b057-1bf7cf5cd1b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9\" (UID: \"886037f3-28bf-47d6-b057-1bf7cf5cd1b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:25.829650 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.829628 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/886037f3-28bf-47d6-b057-1bf7cf5cd1b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9\" (UID: \"886037f3-28bf-47d6-b057-1bf7cf5cd1b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:25.852919 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.852892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:25.997497 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:25.997470 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9"] Jul 27 17:52:25.998927 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:52:25.998902 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886037f3_28bf_47d6_b057_1bf7cf5cd1b7.slice/crio-889e8e416a3e25fc9be37815943e3181751913d3ed50ee772abf758558ad0f72 WatchSource:0}: Error finding container 889e8e416a3e25fc9be37815943e3181751913d3ed50ee772abf758558ad0f72: Status 404 returned error can't find the container with id 889e8e416a3e25fc9be37815943e3181751913d3ed50ee772abf758558ad0f72 Jul 27 17:52:26.530959 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:26.530864 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" event={"ID":"886037f3-28bf-47d6-b057-1bf7cf5cd1b7","Type":"ContainerStarted","Data":"fad9e0447cf1ec2c0869cdbe01d67a07fba3375dacddf91e3753731507d4a923"} Jul 27 17:52:26.530959 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:26.530926 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" event={"ID":"886037f3-28bf-47d6-b057-1bf7cf5cd1b7","Type":"ContainerStarted","Data":"889e8e416a3e25fc9be37815943e3181751913d3ed50ee772abf758558ad0f72"} Jul 27 17:52:26.551266 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:26.551209 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" podStartSLOduration=1.551191136 podStartE2EDuration="1.551191136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 17:52:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:52:26.55012333 +0000 UTC m=+3352.834282613" watchObservedRunningTime="2026-07-27 17:52:26.551191136 +0000 UTC m=+3352.835350418" Jul 27 17:52:27.538430 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:27.538366 2582 generic.go:356] "Generic (PLEG): container finished" podID="886037f3-28bf-47d6-b057-1bf7cf5cd1b7" containerID="fad9e0447cf1ec2c0869cdbe01d67a07fba3375dacddf91e3753731507d4a923" exitCode=0 Jul 27 17:52:27.538430 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:27.538414 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" event={"ID":"886037f3-28bf-47d6-b057-1bf7cf5cd1b7","Type":"ContainerDied","Data":"fad9e0447cf1ec2c0869cdbe01d67a07fba3375dacddf91e3753731507d4a923"} Jul 27 17:52:28.684423 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:28.684391 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:28.759169 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:28.759135 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/886037f3-28bf-47d6-b057-1bf7cf5cd1b7-maas-api-service-ca\") pod \"886037f3-28bf-47d6-b057-1bf7cf5cd1b7\" (UID: \"886037f3-28bf-47d6-b057-1bf7cf5cd1b7\") " Jul 27 17:52:28.759511 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:28.759490 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/886037f3-28bf-47d6-b057-1bf7cf5cd1b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "886037f3-28bf-47d6-b057-1bf7cf5cd1b7" (UID: "886037f3-28bf-47d6-b057-1bf7cf5cd1b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 17:52:28.859919 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:28.859885 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/886037f3-28bf-47d6-b057-1bf7cf5cd1b7-maas-api-service-ca\") on node \"ip-10-0-135-54.ec2.internal\" DevicePath \"\"" Jul 27 17:52:29.551786 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:29.551755 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" Jul 27 17:52:29.551982 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:29.551754 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9" event={"ID":"886037f3-28bf-47d6-b057-1bf7cf5cd1b7","Type":"ContainerDied","Data":"889e8e416a3e25fc9be37815943e3181751913d3ed50ee772abf758558ad0f72"} Jul 27 17:52:29.551982 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:29.551886 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="889e8e416a3e25fc9be37815943e3181751913d3ed50ee772abf758558ad0f72" Jul 27 17:52:36.159945 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:36.159902 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh"] Jul 27 17:52:36.166513 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:36.166477 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-509a5d-29752905-lnrmh"] Jul 27 17:52:36.202514 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:36.202480 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1990f98-81d9-4fa4-995c-dc29438ada8c" path="/var/lib/kubelet/pods/b1990f98-81d9-4fa4-995c-dc29438ada8c/volumes" Jul 27 17:52:40.637202 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:40.637166 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9"] Jul 27 17:52:40.642270 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:40.642244 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-509a5d-lk9n9"] Jul 27 17:52:40.700792 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:40.700756 2582 scope.go:117] "RemoveContainer" containerID="897dd7186d841f6d683cb50e63fc1384ce49bc71d3f62b8943a981d10778addd" Jul 27 17:52:40.710470 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:40.710448 2582 scope.go:117] "RemoveContainer" containerID="e0cedc917e91cdbc432c08f994e918af5e404a2b52630c0391c21084fb6128dd" Jul 27 17:52:42.199590 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:42.199554 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="886037f3-28bf-47d6-b057-1bf7cf5cd1b7" path="/var/lib/kubelet/pods/886037f3-28bf-47d6-b057-1bf7cf5cd1b7/volumes" Jul 27 17:52:48.424884 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:48.424833 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6458d6b58d-chc22_3cb053cd-d164-46d4-ad76-ab0935841b0e/authorino/0.log" Jul 27 17:52:54.527318 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:54.527288 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c484996-4n2zq_5ba683be-eb69-4a31-88bb-682ae04294ae/copy-manifests/0.log" Jul 27 17:52:54.535511 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:54.535479 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c484996-4n2zq_5ba683be-eb69-4a31-88bb-682ae04294ae/manager/0.log" Jul 27 17:52:55.154080 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:55.154029 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57dc978488-w2jb7_7aba2282-7079-4d59-9d46-ee74dce8cce7/maas-api/0.log" Jul 27 17:52:55.494607 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:55.494521 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29752905-pb7ps_826db24b-39e0-4eca-8623-15fec7b43ce7/cleanup/0.log" Jul 27 17:52:56.382287 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.382255 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx_5b4fc03f-d9d3-419a-b7ed-11b203d83b8a/util/0.log" Jul 27 17:52:56.391438 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.391408 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx_5b4fc03f-d9d3-419a-b7ed-11b203d83b8a/pull/0.log" Jul 27 17:52:56.400454 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.400435 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx_5b4fc03f-d9d3-419a-b7ed-11b203d83b8a/extract/0.log" Jul 27 17:52:56.507203 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.507170 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln_6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04/util/0.log" Jul 27 17:52:56.515004 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.514965 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln_6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04/pull/0.log" Jul 27 17:52:56.522711 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.522690 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln_6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04/extract/0.log" Jul 27 17:52:56.632483 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.632389 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc_7ebcb16b-0c9a-45df-b6fc-1c410fa6161e/util/0.log" Jul 27 17:52:56.639170 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.639149 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc_7ebcb16b-0c9a-45df-b6fc-1c410fa6161e/pull/0.log" Jul 27 17:52:56.645951 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.645911 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc_7ebcb16b-0c9a-45df-b6fc-1c410fa6161e/extract/0.log" Jul 27 17:52:56.753860 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.753824 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb_fa03a0ee-2a5a-48d5-86be-1e70e1ec4791/pull/0.log" Jul 27 17:52:56.761021 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.760989 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb_fa03a0ee-2a5a-48d5-86be-1e70e1ec4791/extract/0.log" Jul 27 17:52:56.767991 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.767968 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb_fa03a0ee-2a5a-48d5-86be-1e70e1ec4791/util/0.log" Jul 27 17:52:56.892612 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:56.892517 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6458d6b58d-chc22_3cb053cd-d164-46d4-ad76-ab0935841b0e/authorino/0.log" Jul 27 17:52:57.010968 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:57.010932 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cwcr7_2add960a-7ba7-4222-8f23-1b59be175e99/manager/0.log" Jul 27 17:52:57.119196 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:57.119167 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8lbcz_50a68335-f97e-4013-b568-544b66449076/manager/0.log" Jul 27 17:52:57.220588 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:57.220514 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9b454_591afeac-d683-4f16-97e4-c59fa0c740cb/kuadrant-console-plugin/0.log" Jul 27 17:52:57.334204 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:57.334175 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gh5mt_0abc3f2a-bb70-4126-a517-279465b4e124/registry-server/0.log" Jul 27 17:52:57.683610 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:57.683575 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qdl6p_66b1081e-5294-42d5-8378-8f023cdb4fc3/manager/0.log" Jul 27 17:52:58.038881 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:58.038787 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb_f488af11-04f3-48ee-ad5c-dbad18742cb8/istio-proxy/0.log" Jul 27 17:52:58.925873 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:58.925843 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79d8f4d6d7-cbq28_66329ae6-c737-4391-a021-514669245bc8/router/0.log" Jul 27 17:52:59.268667 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:59.268579 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-gd7nn_59d8a06c-e478-4ed8-920c-7e48dbbd1e9d/main/0.log" Jul 27 17:52:59.734751 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:52:59.734709 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-47tk5_2926f79c-8d6d-4013-9953-bfe9d962d618/main/0.log" Jul 27 17:53:08.691393 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:08.691340 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kb9bl_ce7d45c4-f8fe-4ae6-87fb-05dd5a5196c9/global-pull-secret-syncer/0.log" Jul 27 17:53:08.881850 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:08.881820 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nk6nm_5bd31b56-43d8-4f43-9a3e-28ffbab38410/konnectivity-agent/0.log" Jul 27 17:53:08.966929 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:08.966841 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-54.ec2.internal_1e5ee8caab48bd3dff1dc3efc23e7297/haproxy/0.log" Jul 27 17:53:13.052862 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.052824 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx_5b4fc03f-d9d3-419a-b7ed-11b203d83b8a/extract/0.log" Jul 27 17:53:13.092202 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.092173 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx_5b4fc03f-d9d3-419a-b7ed-11b203d83b8a/util/0.log" Jul 27 17:53:13.118447 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.118409 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wtrtx_5b4fc03f-d9d3-419a-b7ed-11b203d83b8a/pull/0.log" Jul 27 17:53:13.158905 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.158874 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln_6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04/extract/0.log" Jul 27 17:53:13.198146 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.198118 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln_6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04/util/0.log" Jul 27 17:53:13.228921 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.228885 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0spjln_6c869b39-0ac6-45b6-9ff0-4d1e3eb35f04/pull/0.log" Jul 27 17:53:13.267781 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.267754 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc_7ebcb16b-0c9a-45df-b6fc-1c410fa6161e/extract/0.log" Jul 27 17:53:13.296249 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.296223 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc_7ebcb16b-0c9a-45df-b6fc-1c410fa6161e/util/0.log" Jul 27 17:53:13.337742 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.337709 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vvpsc_7ebcb16b-0c9a-45df-b6fc-1c410fa6161e/pull/0.log" Jul 27 17:53:13.377164 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.377137 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb_fa03a0ee-2a5a-48d5-86be-1e70e1ec4791/extract/0.log" Jul 27 17:53:13.406306 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.406282 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb_fa03a0ee-2a5a-48d5-86be-1e70e1ec4791/util/0.log" Jul 27 17:53:13.437545 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.437510 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nj7fb_fa03a0ee-2a5a-48d5-86be-1e70e1ec4791/pull/0.log" Jul 27 17:53:13.923085 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:13.923051 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6458d6b58d-chc22_3cb053cd-d164-46d4-ad76-ab0935841b0e/authorino/0.log" Jul 27 17:53:14.020173 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:14.020140 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-cwcr7_2add960a-7ba7-4222-8f23-1b59be175e99/manager/0.log" Jul 27 17:53:14.085579 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:14.085546 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8lbcz_50a68335-f97e-4013-b568-544b66449076/manager/0.log" Jul 27 17:53:14.120499 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:14.120471 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9b454_591afeac-d683-4f16-97e4-c59fa0c740cb/kuadrant-console-plugin/0.log" Jul 27 17:53:14.205605 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:14.205503 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gh5mt_0abc3f2a-bb70-4126-a517-279465b4e124/registry-server/0.log" Jul 27 17:53:14.724009 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:14.723970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qdl6p_66b1081e-5294-42d5-8378-8f023cdb4fc3/manager/0.log" Jul 27 17:53:16.438440 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.438410 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/alertmanager/0.log" Jul 27 17:53:16.498798 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.498752 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/config-reloader/0.log" Jul 27 17:53:16.575464 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.575435 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/kube-rbac-proxy-web/0.log" Jul 27 17:53:16.640927 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.640899 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/kube-rbac-proxy/0.log" Jul 27 17:53:16.674455 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.674429 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/kube-rbac-proxy-metric/0.log" Jul 27 17:53:16.711492 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.711374 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/prom-label-proxy/0.log" Jul 27 17:53:16.765570 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.765515 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4703f69c-24cc-47fa-bc3e-32000223c839/init-config-reloader/0.log" Jul 27 17:53:16.877265 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.877231 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4f75x_60a05bc9-3b07-4582-955a-0b0bc99befa8/kube-state-metrics/0.log" Jul 27 17:53:16.918084 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.918047 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4f75x_60a05bc9-3b07-4582-955a-0b0bc99befa8/kube-rbac-proxy-main/0.log" Jul 27 17:53:16.956252 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.956224 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-4f75x_60a05bc9-3b07-4582-955a-0b0bc99befa8/kube-rbac-proxy-self/0.log" Jul 27 17:53:16.997786 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:16.997689 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6595d7555b-wsmnr_7221b0b2-63bd-4318-94db-d343486615fa/metrics-server/0.log" Jul 27 17:53:17.044018 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.043991 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-mrpm6_9241cf42-09bc-4048-8525-fb5811686605/monitoring-plugin/0.log" Jul 27 17:53:17.354099 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.354061 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq2xq_fbeea8c0-3870-4e06-aba6-e41473363130/node-exporter/0.log" Jul 27 17:53:17.407670 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.407633 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq2xq_fbeea8c0-3870-4e06-aba6-e41473363130/kube-rbac-proxy/0.log" Jul 27 17:53:17.500864 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.500838 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sq2xq_fbeea8c0-3870-4e06-aba6-e41473363130/init-textfile/0.log" Jul 27 17:53:17.647057 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.646981 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-wk8vj_4c289849-4cb0-4cc7-8ccf-93097d7f1946/kube-rbac-proxy-main/0.log" Jul 27 17:53:17.675637 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.675594 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-wk8vj_4c289849-4cb0-4cc7-8ccf-93097d7f1946/kube-rbac-proxy-self/0.log" Jul 27 17:53:17.711974 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:17.711943 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-wk8vj_4c289849-4cb0-4cc7-8ccf-93097d7f1946/openshift-state-metrics/0.log" Jul 27 17:53:18.406396 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.406353 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn"] Jul 27 17:53:18.406853 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.406838 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886037f3-28bf-47d6-b057-1bf7cf5cd1b7" containerName="revoke-keys" Jul 27 17:53:18.406901 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.406854 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="886037f3-28bf-47d6-b057-1bf7cf5cd1b7" containerName="revoke-keys" Jul 27 17:53:18.406939 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.406933 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="886037f3-28bf-47d6-b057-1bf7cf5cd1b7" containerName="revoke-keys" Jul 27 17:53:18.410151 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.410132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.420691 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.420670 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tn87g\"/\"openshift-service-ca.crt\"" Jul 27 17:53:18.420945 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.420929 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tn87g\"/\"default-dockercfg-78w79\"" Jul 27 17:53:18.421036 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.421009 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tn87g\"/\"kube-root-ca.crt\"" Jul 27 17:53:18.436520 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.436487 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-lib-modules\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.436711 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.436571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-proc\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.436711 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.436634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-podres\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.436711 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.436661 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-sys\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.436711 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.436696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbq8d\" (UniqueName: \"kubernetes.io/projected/2efb9d5e-768d-4e44-9172-73c14327a485-kube-api-access-kbq8d\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.437052 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.436981 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn"] Jul 27 17:53:18.513722 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.513688 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f9b5986d6-5g7jg_c2526be3-7e0c-4185-af23-9ce719dc42e5/telemeter-client/0.log" Jul 27 17:53:18.537633 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537581 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-proc\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.537817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-podres\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.537817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537677 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-sys\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.537817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbq8d\" (UniqueName: \"kubernetes.io/projected/2efb9d5e-768d-4e44-9172-73c14327a485-kube-api-access-kbq8d\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.537817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537706 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-proc\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.537817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537733 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-lib-modules\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.537817 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-sys\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.538023 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-podres\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.538023 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.537817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2efb9d5e-768d-4e44-9172-73c14327a485-lib-modules\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.551147 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.551110 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbq8d\" (UniqueName: \"kubernetes.io/projected/2efb9d5e-768d-4e44-9172-73c14327a485-kube-api-access-kbq8d\") pod \"perf-node-gather-daemonset-25ssn\" (UID: \"2efb9d5e-768d-4e44-9172-73c14327a485\") " pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.556985 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.556957 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f9b5986d6-5g7jg_c2526be3-7e0c-4185-af23-9ce719dc42e5/reload/0.log" Jul 27 17:53:18.615994 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.615966 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f9b5986d6-5g7jg_c2526be3-7e0c-4185-af23-9ce719dc42e5/kube-rbac-proxy/0.log" Jul 27 17:53:18.720600 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.720514 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:18.893289 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:18.893266 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn"] Jul 27 17:53:18.895277 ip-10-0-135-54 kubenswrapper[2582]: W0727 17:53:18.895248 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2efb9d5e_768d_4e44_9172_73c14327a485.slice/crio-3ed74ff07f9c5749f3bccdd89a947dabe9e4085624c82f13009efefb27976fe8 WatchSource:0}: Error finding container 3ed74ff07f9c5749f3bccdd89a947dabe9e4085624c82f13009efefb27976fe8: Status 404 returned error can't find the container with id 3ed74ff07f9c5749f3bccdd89a947dabe9e4085624c82f13009efefb27976fe8 Jul 27 17:53:19.862183 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:19.862148 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" event={"ID":"2efb9d5e-768d-4e44-9172-73c14327a485","Type":"ContainerStarted","Data":"9ac1c1fbe4572b913d9865217fdcf6156da0c5c751fdcb0da838161d67285e4b"} Jul 27 17:53:19.862855 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:19.862202 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" event={"ID":"2efb9d5e-768d-4e44-9172-73c14327a485","Type":"ContainerStarted","Data":"3ed74ff07f9c5749f3bccdd89a947dabe9e4085624c82f13009efefb27976fe8"} Jul 27 17:53:19.862855 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:19.862222 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:19.895587 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:19.895536 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" podStartSLOduration=1.8955191519999999 podStartE2EDuration="1.895519152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 17:53:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 17:53:19.894849059 +0000 UTC m=+3406.179008341" watchObservedRunningTime="2026-07-27 17:53:19.895519152 +0000 UTC m=+3406.179678435" Jul 27 17:53:20.098342 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:20.098310 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-dnr46_3cb439a8-fda2-463a-8abb-75b3bb645314/networking-console-plugin/0.log" Jul 27 17:53:21.464342 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:21.464313 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66cf8c55bd-2swwx_2462602f-96e0-4173-95e6-656585f9d5e5/console/0.log" Jul 27 17:53:21.534065 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:21.534035 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-47rzq_055c4bbc-0d60-4538-b1d4-9f54225ff3a2/download-server/0.log" Jul 27 17:53:22.236221 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:22.236189 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-2lbft_2b5f0b6d-2ba7-487f-8594-a2a6e9e5aa85/volume-data-source-validator/0.log" Jul 27 17:53:23.375946 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:23.375916 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-567dl_a906bcd2-a953-4dc1-802b-ea4f840b3bcb/dns/0.log" Jul 27 17:53:23.424451 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:23.424423 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-567dl_a906bcd2-a953-4dc1-802b-ea4f840b3bcb/kube-rbac-proxy/0.log" Jul 27 17:53:23.632979 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:23.632898 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhnf5_d9b633ee-da5c-4a20-98d1-ad5688caf314/dns-node-resolver/0.log" Jul 27 17:53:24.333496 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:24.333441 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b8c8d4fdc-xhswd_7eff295e-1fad-4966-9d70-769c896d0561/registry/0.log" Jul 27 17:53:24.457758 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:24.457726 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5gbj_cd1e1b48-2cc3-4e96-9bec-a23842e66afb/node-ca/0.log" Jul 27 17:53:25.500599 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:25.500565 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb_f488af11-04f3-48ee-ad5c-dbad18742cb8/istio-proxy/0.log" Jul 27 17:53:25.878762 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:25.878733 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tn87g/perf-node-gather-daemonset-25ssn" Jul 27 17:53:26.095342 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:26.095297 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79d8f4d6d7-cbq28_66329ae6-c737-4391-a021-514669245bc8/router/0.log" Jul 27 17:53:26.961710 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:26.961667 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-29696_3df2fcdf-403d-4c07-9337-29ec40b859bd/serve-healthcheck-canary/0.log" Jul 27 17:53:27.741955 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:27.741909 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bxswn_c9d28086-fbfc-4d08-9d3d-1db5193d3c1f/insights-operator/0.log" Jul 27 17:53:27.746041 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:27.746015 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-bxswn_c9d28086-fbfc-4d08-9d3d-1db5193d3c1f/insights-operator/1.log" Jul 27 17:53:27.784399 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:27.784352 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8w66f_e90fe999-65f1-4595-96fb-baeceab626ba/kube-rbac-proxy/0.log" Jul 27 17:53:27.814685 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:27.814658 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8w66f_e90fe999-65f1-4595-96fb-baeceab626ba/exporter/0.log" Jul 27 17:53:27.844083 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:27.844049 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8w66f_e90fe999-65f1-4595-96fb-baeceab626ba/extractor/0.log" Jul 27 17:53:28.330498 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:28.330473 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb_f488af11-04f3-48ee-ad5c-dbad18742cb8/istio-proxy/0.log" Jul 27 17:53:28.811890 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:28.811860 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79d8f4d6d7-cbq28_66329ae6-c737-4391-a021-514669245bc8/router/0.log" Jul 27 17:53:30.138531 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:30.138499 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wgpsb_f488af11-04f3-48ee-ad5c-dbad18742cb8/istio-proxy/0.log" Jul 27 17:53:30.660204 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:30.660153 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79d8f4d6d7-cbq28_66329ae6-c737-4391-a021-514669245bc8/router/0.log" Jul 27 17:53:31.341999 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:31.341969 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c484996-4n2zq_5ba683be-eb69-4a31-88bb-682ae04294ae/manager/0.log" Jul 27 17:53:31.373041 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:31.373015 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c484996-4n2zq_5ba683be-eb69-4a31-88bb-682ae04294ae/copy-manifests/0.log" Jul 27 17:53:32.853179 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:32.853148 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-b56999664-2r4kj_afbc6d9f-d48f-48bb-b211-9c91126981d0/manager/0.log" Jul 27 17:53:32.911592 ip-10-0-135-54 kubenswrapper[2582]: I0727 17:53:32.911547 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-hpss9_4f947712-1b29-4ee2-a2b4-3b05c2cac568/openshift-lws-operator/0.log"