Jul 24 14:20:22.879960 ip-10-0-135-93 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 14:20:22.879971 ip-10-0-135-93 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 14:20:22.879980 ip-10-0-135-93 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 14:20:22.880299 ip-10-0-135-93 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 14:20:33.026017 ip-10-0-135-93 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 14:20:33.026037 ip-10-0-135-93 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f90c8aa605ea420c9c37cb0755b117a9 --
Jul 24 14:22:21.478151 ip-10-0-135-93 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 14:22:21.782224 ip-10-0-135-93 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:22:21.782224 ip-10-0-135-93 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 14:22:21.782224 ip-10-0-135-93 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:22:21.782224 ip-10-0-135-93 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 14:22:21.782224 ip-10-0-135-93 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 14:22:21.783000 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.782865    2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 14:22:21.788810 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788793    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:22:21.788810 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788809    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788814    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788818    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788821    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788825    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788828    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788831    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788834    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788838    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788840    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788843    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788846    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788848    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788851    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788853    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788856    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788858    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788861    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788863    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788866    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:22:21.788874 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788869    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788871    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788874    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788877    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788879    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788882    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788885    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788890    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788893    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788895    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788898    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788900    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788903    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788905    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788908    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788910    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788913    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788916    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788919    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788921    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:22:21.789381 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788924    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788927    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788929    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788931    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788935    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788937    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788940    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788944    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788947    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788950    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788953    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788956    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788958    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788961    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788964    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788967    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788970    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788972    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788975    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:22:21.789890 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788978    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788981    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788983    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788986    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788988    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788991    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788993    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.788997    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789000    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789002    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789005    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789008    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789010    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789013    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789015    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789018    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789021    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789023    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789025    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789028    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:22:21.790355 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789030    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789033    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789036    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789038    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789040    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789043    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789047    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789050    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789053    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789056    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789058    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789061    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789063    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789066    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789068    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789071    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789075    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789078    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789082    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:22:21.790850 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789769    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789776    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789780    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789783    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789786    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789790    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789792    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789796    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789798    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789801    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789804    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789806    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789809    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789812    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789814    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789817    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789820    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789822    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789825    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789828    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:22:21.791310 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789831    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789834    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789836    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789839    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789842    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789845    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789848    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789850    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789853    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789855    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789858    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789860    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789863    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789865    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789868    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789871    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789873    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789875    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789878    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789881    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:22:21.791826 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789883    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789886    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789888    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789891    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789893    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789896    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789899    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789901    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789904    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789906    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789910    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789913    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789916    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789919    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789921    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789924    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789927    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789929    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789932    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789934    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:22:21.792306 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789938    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789942    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789945    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789947    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789950    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789954    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789957    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789960    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789963    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789965    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789968    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789971    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789973    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789976    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789978    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789981    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789983    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789986    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789988    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:22:21.792799 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789991    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789993    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.789997    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790001    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790004    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790008    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790011    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790013    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790016    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790018    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790021    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790024    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790027    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790029    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790031    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790034    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790036    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790039    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790042    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790044    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:22:21.793273 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790396    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790400    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790403    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790406    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790409    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790412    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790415    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790418    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790421    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790423    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790426    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790429    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790432    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790435    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790437    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790440    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790443    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790446    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790449    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790451    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:22:21.793778 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790454    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790456    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790459    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790462    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790464    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790467    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790470    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790472    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790477    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790480    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790482    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790485    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790488    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790490    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790494    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790496    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790499    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790502    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790505    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790507    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:22:21.794249 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790510    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790512    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790515    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790517    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790520    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790522    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790525    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790528    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790531    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790547    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790550    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790553    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790555    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790560    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790564    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790567    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790570    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790573    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790576    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:22:21.794777 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790579    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790583    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790586    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790589    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790591    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790594    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790598    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790602    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790605    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790608    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790611    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790615    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790617    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790620    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790625    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790628    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790631    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790634    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790636    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:22:21.795229 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790639    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790642    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790644    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790647    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790650    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790652    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790655    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790657    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790660    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790663    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790666    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790668    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790671    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790674    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790678    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790680    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790683    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790686    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790688    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790691    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:22:21.795748 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.790694    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790763    2584 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790769    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790778    2584 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790782    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790787    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790790    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790794    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790798    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790802    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790805    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790808    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790812    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790815    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790818    2584 flags.go:64] FLAG: --cgroup-root=""
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790821    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790824    2584 flags.go:64] FLAG: --client-ca-file=""
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790827    2584 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790830    2584 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790835    2584 flags.go:64] FLAG: --cluster-domain=""
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790838    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790841    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790844    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790847    2584 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 14:22:21.796222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790851    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790855    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790859    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790863    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790866    2584 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790869    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790873    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790876    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790879    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790883    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790886    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790889    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790892    2584 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790895    2584 flags.go:64] FLAG: --enable-server="true"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.790898    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791450    2584 flags.go:64] FLAG: --event-burst="100"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791454    2584 flags.go:64] FLAG: --event-qps="50"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791458    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791461    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791464    2584 flags.go:64] FLAG: --eviction-hard=""
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791468    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791473    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791476    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791479    2584 flags.go:64] FLAG: --eviction-soft=""
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791491    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 14:22:21.796809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791495    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791499    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791502    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791505    2584 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791508    2584 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791510    2584 flags.go:64] FLAG: --feature-gates=""
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791514    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791518    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791521    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791525    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791528    2584 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791531    2584 flags.go:64] FLAG: --help="false"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791548    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791551    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791554    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791557    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791561    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791564    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791567    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791570    2584 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791573    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791576    2584 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791580    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791583    2584 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 14:22:21.797426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791586    2584 flags.go:64] FLAG: --kube-reserved=""
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791589    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791591    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791594    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791600    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791603    2584 flags.go:64] FLAG: --lock-file=""
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791606    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791609    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791612    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791618    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791621    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791624    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791627    2584 flags.go:64] FLAG: --logging-format="text"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791630    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791633    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791636    2584 flags.go:64] FLAG: --manifest-url=""
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791639    2584 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791643    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791646    2584 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791650    2584 flags.go:64] FLAG: --max-pods="110"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791653    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791657    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791666    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791670    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791673    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 14:22:21.798182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791676    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791679    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791686    2584 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791689    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791692    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791695    2584 flags.go:64] FLAG: --pod-cidr=""
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791700    2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791710    2584 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791713    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791717    2584 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791719    2584 flags.go:64] FLAG: --port="10250"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791723    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791727    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00e3ba41b08a45dec"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791731    2584 flags.go:64] FLAG: --qos-reserved=""
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791734    2584 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791737    2584 flags.go:64] FLAG: --register-node="true"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791740    2584 flags.go:64] FLAG: --register-with-taints=""
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791747    2584 flags.go:64] FLAG: --registry-burst="10"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791750    2584 flags.go:64] FLAG: --registry-qps="5"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791753    2584 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791756    2584 flags.go:64] FLAG: --reserved-memory=""
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791760    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791763    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791766    2584 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791769    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 14:22:21.799248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791772    2584 flags.go:64] FLAG: --runonce="false"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791775    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791778    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791781    2584 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791784    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791786    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791790    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791793    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791796    2584 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791799    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791802    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791804    2584 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791807    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791811    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791819    2584 flags.go:64] FLAG: --system-cgroups=""
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791822    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791828    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791831    2584 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791834    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791844    2584 flags.go:64] FLAG: --tls-min-version=""
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791847    2584 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791850    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791853    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791856    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791859    2584 flags.go:64] FLAG: --v="2"
Jul 24 14:22:21.800376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791863    2584 flags.go:64] FLAG: --version="false"
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791867    2584 flags.go:64] FLAG: --vmodule=""
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791870    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.791874    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.791999    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792004    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792007    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792010    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792013    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792016    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792019    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792021    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792024    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792027    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792029    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792032    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792035    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792037    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792040    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792043    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:22:21.801481 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792045    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792048    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792051    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792054    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792056    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792059    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792063    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792065    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792068    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792072    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792074    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792077    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792080    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792082    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792085    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792088    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792091    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792094    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792096    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792099    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:22:21.802126 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792101    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792104    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792106    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792109    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792111    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792114    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792116    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792119    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792121    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792124    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792126    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792129    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792132    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792134    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792137    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792140    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792142    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792145    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792149    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792152    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:22:21.802962 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792155    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792159    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792161    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792164    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792167    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792169    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792172    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792175    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792177    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792180    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792182    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792185    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792188    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792191    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792193    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792196    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792198    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792201    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792203    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792206    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:22:21.803796 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792209    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792211    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792214    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792216    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792219    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792222    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792226    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792229    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792232    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792236    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792241    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792244    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792247    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792251    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792254    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792257    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792260    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792263    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792266    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:22:21.804372 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792269    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792272    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792274    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.792277    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.792807    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.800049    2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.800067    2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.800094    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.800103    2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800185    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800193    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800198    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800203    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800208    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800212    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:22:21.805055 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800217    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800221    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800229    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800235    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800240    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800244    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800249    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800254    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800258    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800263    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800267    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800271    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800275    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800281    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800285    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800289    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800294    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800298    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800303    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:22:21.805435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800308    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800313    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800317    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800322    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800328    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800332    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800336    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800341    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800346    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800350    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800354    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800359    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800363    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800367    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800371    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800375    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800379    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800383    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800387    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800392    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:22:21.805916 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800396    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800400    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800404    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800409    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800413    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800417    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800422    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800426    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800430    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800435    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800439    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800444    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800448    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800452    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800457    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800462    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800466    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800471    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800475    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800479    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:22:21.806377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800483    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800487    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800491    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800496    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800500    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800504    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800508    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800512    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800516    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800520    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800525    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800528    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800550    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800554    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800560    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800565    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800569    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800576    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800582    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:22:21.806879 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800587    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800591    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800596    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800601    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800605    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800609    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800613    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800618    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800623    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800628    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800632    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800637    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800641    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800646    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800650    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:22:21.807327 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.800657    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800836    2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800844    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800849    2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800854    2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800858    2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800863    2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800867    2584 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800872    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800876    2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800880    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800884    2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800888    2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800893    2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800897    2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800901    2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800905    2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800910    2584 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800914    2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800918    2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 14:22:21.807755 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800923    2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800927    2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800931    2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800935    2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800939    2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800944    2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800949    2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800953    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800957    2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800963    2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800967    2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800971    2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800975    2584 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800979    2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800983    2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800987    2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800991    2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.800996    2584 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801000    2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801007    2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 14:22:21.808245 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801012    2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801017    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801022    2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801028    2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801033    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801037    2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801042    2584 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801046    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801051    2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801055    2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801059    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801063    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801067    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801071    2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801078    2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801083    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801089    2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801094    2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801098    2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 14:22:21.808739 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801103    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801107    2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801112    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801116    2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801121    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801125    2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801129    2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801135    2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801140    2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801144    2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801148    2584 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801152    2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801156    2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801160    2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801164    2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801169    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801173    2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801178    2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801182    2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 14:22:21.809201 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801186    2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801190    2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801194    2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801198    2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801202    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801207    2584 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801211    2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801215    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801219    2584 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801223    2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801227    2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801232    2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801236    2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801240    2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801244    2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801248    2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801252    2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801257    2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801261    2584 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801266    2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 14:22:21.809737 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801270    2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:21.801274    2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.801281    2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.801439    2584 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.804680    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.805908    2584 server.go:1019] "Starting client certificate rotation"
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.806002    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.806032    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.807974    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 14:22:21.810213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.809047    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 14:22:21.819176 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.819157    2584 log.go:25] "Validated CRI v1 runtime API"
Jul 24 14:22:21.821184 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.821168    2584 log.go:25] "Validated CRI v1 image API"
Jul 24 14:22:21.822028 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.822003    2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 14:22:21.824056 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.824038    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f3b74de-a106-4a0a-bc18-80ee297292fd:/dev/nvme0n1p3 c1ca202c-4146-4362-a57b-d78d6d476a23:/dev/nvme0n1p4]
Jul 24 14:22:21.824120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.824056    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 14:22:21.829714 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.829587    2584 manager.go:216] Machine: {Timestamp:2026-07-24 14:22:21.828324193 +0000 UTC m=+0.257322098 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107296 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fec4cb0519376c37ec37103f3ae7c SystemUUID:ec2fec4c-b051-9376-c37e-c37103f3ae7c BootID:f90c8aa6-05ea-420c-9c37-cb0755b117a9 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:60:e1:fc:04:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:60:e1:fc:04:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:61:9a:09:dc:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 14:22:21.829714 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.829708    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 14:22:21.829808 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.829775    2584 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 14:22:21.832264 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.832236    2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 14:22:21.832385 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.832266    2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 14:22:21.832908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.832898    2584 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 14:22:21.832939 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.832910    2584 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 14:22:21.832939 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.832917    2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 14:22:21.832939 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.832930    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 14:22:21.833740 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.833721    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 14:22:21.834220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.834209    2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 14:22:21.838454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838443    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 14:22:21.838612 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838602    2584 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 14:22:21.838653 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838632    2584 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 14:22:21.838653 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838652    2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 14:22:21.838714 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838674    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 14:22:21.838714 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838686    2584 kubelet.go:398] "Adding apiserver pod source"
Jul 24 14:22:21.838714 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.838697    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 14:22:21.839253 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.839238    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 14:22:21.839317 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.839259    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 14:22:21.841100 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.841084    2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 14:22:21.844057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.844038    2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 14:22:21.844131 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.844088    2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 14:22:21.845586 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845573    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845591    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845597    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845603    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845611    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845617    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845625    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845630    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 14:22:21.845635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845637    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 14:22:21.845835 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845643    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 14:22:21.845835 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845651    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 14:22:21.845835 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.845658    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 14:22:21.846265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.846254    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 14:22:21.846265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.846265    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 14:22:21.849528 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.849512    2584 server.go:1268] "Started kubelet"
Jul 24 14:22:21.849692 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.849644    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 14:22:21.849762 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.849675    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:21.849762 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.849716    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 14:22:21.849861 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.849672    2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 14:22:21.850155 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.850127    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:22:21.850222 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.850139    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:22:21.850467 ip-10-0-135-93 systemd[1]: Started Kubernetes Kubelet.
Jul 24 14:22:21.851152 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.851137    2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 14:22:21.851905 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.851890    2584 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 14:22:21.856232 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.855486    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4cf7cf3bf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.849482175 +0000 UTC m=+0.278480089,LastTimestamp:2026-07-24 14:22:21.849482175 +0000 UTC m=+0.278480089,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:21.856369 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.856354    2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 14:22:21.858331 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858318    2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 14:22:21.858392 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858331    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 14:22:21.858448 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.858429    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:22:21.858875 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858860    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 14:22:21.858875 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858867    2584 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 14:22:21.858996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858885    2584 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 14:22:21.858996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858966    2584 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 14:22:21.858996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.858974    2584 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 14:22:21.859113 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859005    2584 factory.go:55] Registering systemd factory
Jul 24 14:22:21.859113 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859016    2584 factory.go:223] Registration of the systemd container factory successfully
Jul 24 14:22:21.859244 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859218    2584 factory.go:153] Registering CRI-O factory
Jul 24 14:22:21.859244 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859232    2584 factory.go:223] Registration of the crio container factory successfully
Jul 24 14:22:21.859390 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859283    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 14:22:21.859390 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859307    2584 factory.go:103] Registering Raw factory
Jul 24 14:22:21.859390 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859321    2584 manager.go:1196] Started watching for new ooms in manager
Jul 24 14:22:21.859684 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.859665    2584 manager.go:349] Starting recovery of all containers
Jul 24 14:22:21.860935 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.860892    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:22:21.861101 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.860986    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 14:22:21.864118 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.864082    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-98vkc"
Jul 24 14:22:21.869736 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.869594    2584 manager.go:354] Recovery completed
Jul 24 14:22:21.871849 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.871827    2584 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 24 14:22:21.875919 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.875907    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:21.878858 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.878841    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:21.878926 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.878868    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:21.878926 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.878878    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:21.879373 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.879359    2584 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 14:22:21.879373 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.879372    2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 14:22:21.879508 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.879389    2584 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 14:22:21.879508 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.879400    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:21.880285 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.880270    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:21.880349 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.880298    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:21.880349 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.880313    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:21.882150 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.882132    2584 policy_none.go:49] "None policy: Start"
Jul 24 14:22:21.882223 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.882155    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 14:22:21.882223 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.882170    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 14:22:21.884169 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.884158    2584 policy_none.go:47] "Start"
Jul 24 14:22:21.887637 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.887576    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:21.889787 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.889730    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:21.896604 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.896522    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:21.907121 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.907062    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:21.8802836 +0000 UTC m=+0.309281510,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:21.918239 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.918175    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:21.880305231 +0000 UTC m=+0.309303140,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:21.920065 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920053    2584 manager.go:337] "Starting Device Plugin manager"
Jul 24 14:22:21.920130 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.920086    2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 14:22:21.920130 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920097    2584 server.go:87] "Starting device plugin registration server"
Jul 24 14:22:21.920335 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920323    2584 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 14:22:21.920409 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920336    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 14:22:21.920449 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920424    2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 14:22:21.920522 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920506    2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 14:22:21.920522 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:21.920520    2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 14:22:21.921020 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.921002    2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 14:22:21.921115 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:21.921053    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:22:23.571119 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.571063    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:21.880318653 +0000 UTC m=+0.309316561,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.582183 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.582120    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc535d33ec0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:23.566413504 +0000 UTC m=+0.351113949,LastTimestamp:2026-07-24 14:22:23.566413504 +0000 UTC m=+0.351113949,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.662964 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.662937    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 14:22:23.665060 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.664044    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 14:22:23.665060 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.664082    2584 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 14:22:23.665060 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.664102    2584 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 14:22:23.665060 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.664131    2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 14:22:23.665060 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.664695    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:23.665740 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.665720    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:23.665825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.665753    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:23.665825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.665767    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:23.665825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.665792    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.673613 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.673593    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:22:23.673613 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.673546    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:23.665738884 +0000 UTC m=+0.450439343,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.676308 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.676289    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.676368 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.676312    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:23.665759847 +0000 UTC m=+0.450460306,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.683663 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.683569    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:23.665773221 +0000 UTC m=+0.450473676,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.708601 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.708581    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Jul 24 14:22:23.764699 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.764665    2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"]
Jul 24 14:22:23.764755 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.764737    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:23.765661 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.765646    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:23.765725 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.765673    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:23.765725 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.765685    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:23.768054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768042    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:23.768182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768165    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.768239 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768203    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:23.768815 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768801    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:23.768933 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768823    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:23.768933 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768802    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:23.768933 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768851    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:23.768933 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768868    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:23.768933 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.768883    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:23.771105 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.771088    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.771183 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.771119    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:23.771784 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.771770    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:23.771890 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.771793    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:23.771890 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.771802    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:23.777686 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.777629    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:23.765659793 +0000 UTC m=+0.550360248,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.786106 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.786046    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:23.765678237 +0000 UTC m=+0.550378692,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.792780 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.792764    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.795127 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.795047    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:23.765690286 +0000 UTC m=+0.550390748,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.796715 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.796699    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.803223 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.803168    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:23.768815215 +0000 UTC m=+0.553515670,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.805922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.805903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/74ca4cffd660659a68e7ccdd89a85ecc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal\" (UID: \"74ca4cffd660659a68e7ccdd89a85ecc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.805992 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.805928    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74ca4cffd660659a68e7ccdd89a85ecc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal\" (UID: \"74ca4cffd660659a68e7ccdd89a85ecc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.805992 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.805944    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/feea928f3452f52f53864c7f7d2090cd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-93.ec2.internal\" (UID: \"feea928f3452f52f53864c7f7d2090cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.814174 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.814112    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:23.768827537 +0000 UTC m=+0.553527992,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.823086 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.823027    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:23.768857652 +0000 UTC m=+0.553558107,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.834692 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.834632    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:23.768855809 +0000 UTC m=+0.553556265,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.844635 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.844579    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:23.768875767 +0000 UTC m=+0.553576224,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.853148 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.853097    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:23.768888616 +0000 UTC m=+0.553589072,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.862051 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.861990    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:23.77178186 +0000 UTC m=+0.556482315,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.872688 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.872628    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:23.7717974 +0000 UTC m=+0.556497855,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.877076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.877062    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:23.877883 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.877867    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:23.877947 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.877897    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:23.877947 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.877907    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:23.877947 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.877930    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.884391 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.884341    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d8e23  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878881827 +0000 UTC m=+0.307879732,LastTimestamp:2026-07-24 14:22:23.771805533 +0000 UTC m=+0.556505988,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.889876 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.889855    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.889942 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.889891    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d2c1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878856733 +0000 UTC m=+0.307854641,LastTimestamp:2026-07-24 14:22:23.877883609 +0000 UTC m=+0.662584070,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.901490 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:23.901437    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-93.ec2.internal.18c53fc4d13d6ccd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-93.ec2.internal,UID:ip-10-0-135-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:21.878873293 +0000 UTC m=+0.307871198,LastTimestamp:2026-07-24 14:22:23.877902497 +0000 UTC m=+0.662602952,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:23.906747 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.906732    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/74ca4cffd660659a68e7ccdd89a85ecc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal\" (UID: \"74ca4cffd660659a68e7ccdd89a85ecc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.906794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.906756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74ca4cffd660659a68e7ccdd89a85ecc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal\" (UID: \"74ca4cffd660659a68e7ccdd89a85ecc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.906794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.906772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/feea928f3452f52f53864c7f7d2090cd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-93.ec2.internal\" (UID: \"feea928f3452f52f53864c7f7d2090cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.906884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.906820    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/feea928f3452f52f53864c7f7d2090cd-config\") pod \"kube-apiserver-proxy-ip-10-0-135-93.ec2.internal\" (UID: \"feea928f3452f52f53864c7f7d2090cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.906884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.906847    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74ca4cffd660659a68e7ccdd89a85ecc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal\" (UID: \"74ca4cffd660659a68e7ccdd89a85ecc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:23.906884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:23.906820    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/74ca4cffd660659a68e7ccdd89a85ecc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal\" (UID: \"74ca4cffd660659a68e7ccdd89a85ecc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:24.095656 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.095577    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:24.099499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.099476    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"
Jul 24 14:22:24.119324 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.119295    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Jul 24 14:22:24.290915 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.290888    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:24.291871 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.291850    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:24.291952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.291892    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:24.291952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.291905    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:24.291952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.291931    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:24.310013 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.309995    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:24.383266 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.383202    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:22:24.505134 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.505107    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:24.594525 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.594503    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:24.596236 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.596211    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:24.596328 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.596278    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:24.596328 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.596290    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:24.599409 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:24.599382    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ca4cffd660659a68e7ccdd89a85ecc.slice/crio-c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc WatchSource:0}: Error finding container c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc: Status 404 returned error can't find the container with id c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc
Jul 24 14:22:24.599689 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:22:24.599633    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeea928f3452f52f53864c7f7d2090cd.slice/crio-1f3035b257a0705aa7d7a5a3b499c76ed0f79b7fba2f3da55556aa84e07976aa WatchSource:0}: Error finding container 1f3035b257a0705aa7d7a5a3b499c76ed0f79b7fba2f3da55556aa84e07976aa: Status 404 returned error can't find the container with id 1f3035b257a0705aa7d7a5a3b499c76ed0f79b7fba2f3da55556aa84e07976aa
Jul 24 14:22:24.604303 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.604264    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:22:24.606401 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.606332    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-93.ec2.internal.18c53fc573b30120  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-93.ec2.internal,UID:feea928f3452f52f53864c7f7d2090cd,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\",Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:24.604487968 +0000 UTC m=+1.389188414,LastTimestamp:2026-07-24 14:22:24.604487968 +0000 UTC m=+1.389188414,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:24.616804 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.616737    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc573b3d05e  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\",Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:24.604541022 +0000 UTC m=+1.389241477,LastTimestamp:2026-07-24 14:22:24.604541022 +0000 UTC m=+1.389241477,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:24.639133 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.639075    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:22:24.668282 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.668242    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal" event={"ID":"feea928f3452f52f53864c7f7d2090cd","Type":"ContainerStarted","Data":"1f3035b257a0705aa7d7a5a3b499c76ed0f79b7fba2f3da55556aa84e07976aa"}
Jul 24 14:22:24.669228 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:24.669205    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc"}
Jul 24 14:22:24.773098 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.773073    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:22:24.864933 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.864910    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:22:24.930115 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:24.930048    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Jul 24 14:22:25.110138 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:25.110104    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:25.112812 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:25.112788    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:25.112954 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:25.112825    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:25.112954 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:25.112856    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:25.112954 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:25.112907    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:25.132858 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:25.132814    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:25.507605 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:25.507573    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:26.403016 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.402935    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5de51ecc2  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" in 1.788s (1.788s including waiting). Image size: 278280461 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.393287874 +0000 UTC m=+3.177988335,LastTimestamp:2026-07-24 14:22:26.393287874 +0000 UTC m=+3.177988335,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:26.441403 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.441322    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-93.ec2.internal.18c53fc5e0a8db47  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-93.ec2.internal,UID:feea928f3452f52f53864c7f7d2090cd,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\" in 1.828s (1.828s including waiting). Image size: 317186379 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.432539463 +0000 UTC m=+3.217239910,LastTimestamp:2026-07-24 14:22:26.432539463 +0000 UTC m=+3.217239910,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:26.497119 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.497097    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:26.497216 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.497070    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-93.ec2.internal.18c53fc5e3c27c86  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-93.ec2.internal,UID:feea928f3452f52f53864c7f7d2090cd,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.48455079 +0000 UTC m=+3.269251253,LastTimestamp:2026-07-24 14:22:26.48455079 +0000 UTC m=+3.269251253,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:26.503367 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.503288    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-135-93.ec2.internal.18c53fc5e444da92  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-135-93.ec2.internal,UID:feea928f3452f52f53864c7f7d2090cd,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.493094546 +0000 UTC m=+3.277795001,LastTimestamp:2026-07-24 14:22:26.493094546 +0000 UTC m=+3.277795001,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:26.513543 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.513515    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:22:26.539870 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.539829    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Jul 24 14:22:26.582001 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.581931    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.573384035 +0000 UTC m=+3.358084490,LastTimestamp:2026-07-24 14:22:26.573384035 +0000 UTC m=+3.358084490,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:26.591070 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.590993    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.581615199 +0000 UTC m=+3.366315663,LastTimestamp:2026-07-24 14:22:26.581615199 +0000 UTC m=+3.366315663,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:26.674565 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.674447    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal" event={"ID":"feea928f3452f52f53864c7f7d2090cd","Type":"ContainerStarted","Data":"31324bf2c1d06e47ac58cf876bef30a1cc67639986d088e05fb4cd0ba5541266"}
Jul 24 14:22:26.674565 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.674464    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:26.675376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.675361    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:26.675458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.675386    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:26.675458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.675395    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:26.675647 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.675627    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:26.675765 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.675709    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f"}
Jul 24 14:22:26.675828 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.675783    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:26.676471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.676449    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:26.676531 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.676480    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:26.676531 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.676497    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:26.676641 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.676628    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:26.733116 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.733093    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:26.733881 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.733866    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:26.733973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.733892    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:26.733973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.733906    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:26.733973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.733933    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:26.742395 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.742370    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:22:26.752506 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:26.752483    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:26.902103 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.902070    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:26.903087 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.903071    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:26.903164 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.903102    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:26.903164 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:26.903116    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:27.137980 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:27.137907    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:22:27.503945 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.503870    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:27.676886 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.676868    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:27.677252 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.676954    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:27.688390 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.688372    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:27.688504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.688396    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:27.688504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.688409    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:27.688504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.688458    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:27.688504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.688482    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:27.688504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:27.688495    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:27.688661 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:27.688611    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:27.688724 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:27.688709    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:27.861999 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:27.861930    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:22:28.506100 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:28.506074    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:29.504648 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:29.504601    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:29.751432 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:29.751405    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Jul 24 14:22:29.953438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:29.953374    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:29.954372 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:29.954357    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:29.954443 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:29.954386    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:29.954443 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:29.954397    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:29.954443 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:29.954425    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:29.972278 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:29.972253    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:30.356795 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:30.356721    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:22:30.502722 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:30.502696    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:30.977354 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:30.977326    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:22:31.110219 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:31.110173    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:31.111157 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:31.111138    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:31.111236 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:31.111172    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:31.111236 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:31.111183    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:31.504165 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:31.504140    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:32.504017 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:32.503989    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:32.630329 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:32.630301    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:22:33.502114 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:33.502088    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:33.565718 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:33.565671    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:22:34.040293 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:34.040267    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:22:34.503311 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:34.503236    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:35.510222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:35.510195    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:36.163824 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:36.163797    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:22:36.373317 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:36.373277    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:36.374254 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:36.374236    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:36.374322 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:36.374269    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:36.374322 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:36.374280    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:36.374322 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:36.374309    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:36.391809 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:36.391779    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:36.505524 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:36.505464    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:37.503494 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.503470    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:37.691691 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.691664    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/0.log"
Jul 24 14:22:37.691824 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.691699    2584 generic.go:356] "Generic (PLEG): container finished" podID="74ca4cffd660659a68e7ccdd89a85ecc" containerID="fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f" exitCode=1
Jul 24 14:22:37.691824 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.691732    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerDied","Data":"fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f"}
Jul 24 14:22:37.691925 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.691891    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:37.692665 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.692649    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:37.692722 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.692676    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:37.692722 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:37.692686    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:37.692878 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:37.692865    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:37.704867 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:37.704776    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:37.695629652 +0000 UTC m=+14.480330094,LastTimestamp:2026-07-24 14:22:37.695629652 +0000 UTC m=+14.480330094,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:37.776636 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:37.776559    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.573384035 +0000 UTC m=+3.358084490,LastTimestamp:2026-07-24 14:22:37.7654295 +0000 UTC m=+14.550129964,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:37.786044 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:37.785978    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.581615199 +0000 UTC m=+3.366315663,LastTimestamp:2026-07-24 14:22:37.773640773 +0000 UTC m=+14.558341228,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:38.502704 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.502671    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:38.694825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.694803    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/0.log"
Jul 24 14:22:38.695193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.694856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8"}
Jul 24 14:22:38.695193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.694955    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:38.695725 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.695709    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:38.695793 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.695740    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:38.695793 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:38.695751    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:38.695960 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:38.695948    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:38.718255 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:38.718232    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:22:39.084658 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:39.084632    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:22:39.504324 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:39.504249    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:40.317509 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:40.317464    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:40.318470 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:40.318450    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:40.318555 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:40.318482    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:40.318555 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:40.318492    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:40.505526 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:40.505504    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:40.756167 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:40.756091    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:22:41.504962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:41.504937    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:42.502425 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:42.502392    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:43.174402 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:43.174368    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:22:43.392674 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:43.392639    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:43.393587 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:43.393566    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:43.393702 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:43.393597    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:43.393702 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:43.393607    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:43.393702 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:43.393634    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:43.414696 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:43.414673    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:43.504782 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:43.504709    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:43.566079 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:43.566045    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:22:44.505353 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:44.505319    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:45.506641 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:45.506609    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:45.838821 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:45.838738    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:22:46.507561 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:46.507531    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:47.508733 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:47.508704    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:48.505108 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:48.505079    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:49.509822 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.509790    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:49.711202 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.711175    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/1.log"
Jul 24 14:22:49.711551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.711538    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/0.log"
Jul 24 14:22:49.711597 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.711572    2584 generic.go:356] "Generic (PLEG): container finished" podID="74ca4cffd660659a68e7ccdd89a85ecc" containerID="ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8" exitCode=1
Jul 24 14:22:49.711627 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.711600    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerDied","Data":"ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8"}
Jul 24 14:22:49.711809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.711796    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:49.712379 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.712366    2584 scope.go:117] "RemoveContainer" containerID="fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f"
Jul 24 14:22:49.712702 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.712678    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:49.712797 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.712714    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:49.712797 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.712725    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:49.712948 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:49.712935    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:49.713036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.713023    2584 scope.go:117] "RemoveContainer" containerID="fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f"
Jul 24 14:22:49.722130 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:49.722109    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:22:49.722227 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:49.722129    2584 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_0 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f'" containerID="fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f"
Jul 24 14:22:49.722227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:49.722173    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f"} err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_0 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'fb6b965817e0a9e0b70619d2c72ebb02e071c96b2943aafae0151094d671630f'"
Jul 24 14:22:49.737093 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:49.737016    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:22:50.187546 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:50.187513    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:22:50.415585 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.415556    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:50.416692 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.416671    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:50.416794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.416710    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:50.416794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.416724    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:50.416794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.416759    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:50.434508 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:50.434483    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:50.504873 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.504810    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:50.714377 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:50.714355    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/1.log"
Jul 24 14:22:51.505251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:51.505223    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:52.506802 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:52.506773    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:53.504170 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:53.504139    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:53.566430 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:53.566407    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:22:54.505687 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:54.505656    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:55.505471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:55.505440    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:56.505031 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:56.505001    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:57.197956 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:57.197928    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:22:57.434952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.434930    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:57.437460 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.437444    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:57.437519 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.437484    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:57.437519 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.437497    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:57.437587 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.437524    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:57.456112 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:22:57.456062    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:22:57.504432 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.504413    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:57.797283 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.797245    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:22:57.798172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.798154    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:22:57.798275 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.798184    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:22:57.798275 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:57.798194    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:22:58.504581 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:58.504550    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:22:59.508712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:22:59.508684    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:00.437994 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:00.437963    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:23:00.505663 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:00.505642    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:00.664541 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:00.664517    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:00.665463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:00.665443    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:00.665559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:00.665477    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:00.665559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:00.665487    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:00.665671 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:00.665659    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:00.678005 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:00.677899    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:37.695629652 +0000 UTC m=+14.480330094,LastTimestamp:2026-07-24 14:23:00.666365133 +0000 UTC m=+37.451065599,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:00.748807 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:00.748732    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.573384035 +0000 UTC m=+3.358084490,LastTimestamp:2026-07-24 14:23:00.739285396 +0000 UTC m=+37.523985860,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:00.757887 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:00.757808    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.581615199 +0000 UTC m=+3.366315663,LastTimestamp:2026-07-24 14:23:00.747747187 +0000 UTC m=+37.532447629,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:01.429196 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:01.429163    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:23:01.504093 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.504066    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:01.729859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.729777    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/1.log"
Jul 24 14:23:01.729859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.729827    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93"}
Jul 24 14:23:01.730291 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.729985    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:01.730851 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.730823    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:01.730955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.730868    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:01.730955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:01.730883    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:01.731158 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:01.731143    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:02.275937 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:02.275907    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:23:02.507501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:02.507473    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:03.505157 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:03.505130    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:03.566535 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:03.566491    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:23:04.209581 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:04.209542    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:04.456969 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:04.456929    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:04.458021 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:04.458001    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:04.458104 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:04.458038    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:04.458104 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:04.458054    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:04.458104 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:04.458080    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:04.479722 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:04.479657    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:04.505340 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:04.505313    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:04.669022 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:04.668991    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:23:05.504334 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:05.504305    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:06.503748 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:06.503718    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:07.505170 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:07.505137    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:08.504889 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:08.504861    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:09.506383 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:09.506350    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:10.506732 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:10.506702    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:11.220701 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:11.220671    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:11.479939 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:11.479863    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:11.480850 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:11.480819    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:11.480952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:11.480863    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:11.480952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:11.480873    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:11.480952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:11.480901    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:11.500029 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:11.500007    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:11.500185 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:11.500169    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:12.497289 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.497257    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:12.746012 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.745989    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/2.log"
Jul 24 14:23:12.746350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.746334    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/1.log"
Jul 24 14:23:12.746454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.746370    2584 generic.go:356] "Generic (PLEG): container finished" podID="74ca4cffd660659a68e7ccdd89a85ecc" containerID="ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93" exitCode=1
Jul 24 14:23:12.746454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.746398    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerDied","Data":"ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93"}
Jul 24 14:23:12.746563 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.746500    2584 scope.go:117] "RemoveContainer" containerID="ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8"
Jul 24 14:23:12.746675 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.746660    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:12.747774 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.747495    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:12.747774 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.747590    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:12.747774 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.747609    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:12.747975 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:12.747856    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:12.747975 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:12.747919    2584 scope.go:117] "RemoveContainer" containerID="ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8"
Jul 24 14:23:12.756437 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:12.756414    2584 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_1 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8'" containerID="ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8"
Jul 24 14:23:12.756490 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:12.756456    2584 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_1 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'ace52f0ac16b82e1bba0274b341d06bf2214ecc3a647f955255bc7f877e699f8'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" logger="UnhandledError"
Jul 24 14:23:12.756593 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:12.756578    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:23:12.765060 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:12.764982    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:23:12.756548642 +0000 UTC m=+49.541249096,Count:2,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:13.505792 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:13.505759    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:13.567309 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:13.567284    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:23:13.749470 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:13.749447    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/2.log"
Jul 24 14:23:14.506081 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:14.506051    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:15.507962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:15.507929    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:16.505987 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:16.505958    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:17.505512 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:17.505482    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:18.229200 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:18.229169    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:18.500172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:18.500093    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:18.500924 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:18.500906    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:18.501019 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:18.500938    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:18.501019 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:18.500952    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:18.501019 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:18.500977    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:18.507220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:18.507194    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:18.515344 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:18.515322    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:19.511412 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:19.511379    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:20.503418 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:20.503391    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:21.502664 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:21.502629    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:22.504085 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:22.504051    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:23.506935 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:23.506903    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:23.567401 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:23.567374    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:23:24.506724 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:24.506693    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:24.664746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:24.664707    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:24.667006 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:24.666989    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:24.667096 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:24.667020    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:24.667096 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:24.667033    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:24.667250 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:24.667235    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:24.667429 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:24.667412    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:23:24.679125 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:24.679046    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:23:24.6673774 +0000 UTC m=+61.452077864,Count:3,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:25.241904 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:25.241871    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:25.504221 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:25.504149    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:25.515477 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:25.515457    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:25.517163 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:25.517147    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:25.517237 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:25.517173    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:25.517237 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:25.517183    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:25.517237 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:25.517206    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:25.536808 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:25.536784    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:26.504029 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:26.504001    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:27.505667 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:27.505638    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:28.506327 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:28.506299    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:29.503847 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:29.503799    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:30.509107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:30.509077    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:30.664841 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:30.664797    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:30.665799 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:30.665783    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:30.665914 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:30.665809    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:30.665914 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:30.665820    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:30.666015 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:30.666004    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:31.503445 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:31.503414    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:32.046771 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:32.046727    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:23:32.250622 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:32.250589    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:32.352642 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:32.352566    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:23:32.507343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:32.507315    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:32.537645 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:32.537627    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:32.538514 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:32.538494    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:32.538619 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:32.538532    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:32.538619 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:32.538546    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:32.538619 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:32.538571    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:32.556324 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:32.556297    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:33.504055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:33.504020    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:33.567791 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:33.567760    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:23:34.508597 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:34.508563    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:35.503507 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.503476    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:35.664545 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.664507    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:35.665416 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.665397    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:35.665495 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.665429    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:35.665495 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.665439    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:35.665664 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:35.665650    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:35.676068 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:35.675987    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:37.695629652 +0000 UTC m=+14.480330094,LastTimestamp:2026-07-24 14:23:35.666601821 +0000 UTC m=+72.451302265,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:35.754924 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:35.754810    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.573384035 +0000 UTC m=+3.358084490,LastTimestamp:2026-07-24 14:23:35.743274775 +0000 UTC m=+72.527975238,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:35.766485 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:35.766413    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.581615199 +0000 UTC m=+3.366315663,LastTimestamp:2026-07-24 14:23:35.75127439 +0000 UTC m=+72.535974854,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:35.782017 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.782000    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/2.log"
Jul 24 14:23:35.782103 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.782035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4"}
Jul 24 14:23:35.782144 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.782126    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:35.782788 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.782773    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:35.782881 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.782804    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:35.782881 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.782819    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:35.783048 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:35.783034    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:35.875506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.875481    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:35.876217 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.876202    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:35.876303 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.876231    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:35.876303 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:35.876246    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:36.505748 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:36.505716    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:37.503271 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:37.503241    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:38.503963 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:38.503936    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:38.670191 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:38.670160    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:23:39.263232 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:39.263201    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:39.503950 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:39.503927    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:39.556513 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:39.556496    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:39.557437 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:39.557391    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:39.557506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:39.557451    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:39.557506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:39.557462    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:39.557506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:39.557493    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:39.579583 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:39.579555    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:40.504433 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:40.504407    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:41.367809 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:41.367771    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:23:41.505456 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:41.505424    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:42.504309 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:42.504278    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:43.503741 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:43.503714    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:43.568396 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:43.568368    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:23:44.503972 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:44.503945    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:45.507848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:45.507720    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:46.272745 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.272707    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:46.506487 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.506457    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:46.580024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.579957    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:46.580890 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.580874    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:46.580973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.580903    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:46.580973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.580920    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:46.580973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.580969    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:46.607627 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.607601    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:46.799399 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.799379    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/3.log"
Jul 24 14:23:46.799727 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.799710    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/2.log"
Jul 24 14:23:46.799826 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.799747    2584 generic.go:356] "Generic (PLEG): container finished" podID="74ca4cffd660659a68e7ccdd89a85ecc" containerID="b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4" exitCode=1
Jul 24 14:23:46.799826 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.799783    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerDied","Data":"b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4"}
Jul 24 14:23:46.799970 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.799897    2584 scope.go:117] "RemoveContainer" containerID="ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93"
Jul 24 14:23:46.800023 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.800014    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:46.800987 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.800886    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:46.800987 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.800921    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:46.800987 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.800933    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:46.801109 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.801097    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:46.801147 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:46.801143    2584 scope.go:117] "RemoveContainer" containerID="ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93"
Jul 24 14:23:46.809215 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.809191    2584 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_2 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93'" containerID="ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93"
Jul 24 14:23:46.809289 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.809227    2584 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_2 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'ff42edf7553aff121ecabffc0a9de426434666484eadc114196830155969bd93'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" logger="UnhandledError"
Jul 24 14:23:46.809373 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.809355    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:23:46.823183 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:46.823113    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:23:46.809318656 +0000 UTC m=+83.594019114,Count:4,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:23:47.504218 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:47.504191    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:47.802148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:47.802122    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/3.log"
Jul 24 14:23:48.503780 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:48.503747    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:49.504622 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:49.504438    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:50.504905 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:50.504877    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:51.505521 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:51.505492    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:52.503935 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:52.503904    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:53.283078 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:53.283038    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:23:53.505179 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:53.505148    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:53.568971 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:53.568890    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:23:53.608361 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:53.608339    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:53.609153 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:53.609138    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:53.609247 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:53.609172    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:53.609247 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:53.609187    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:53.609247 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:53.609220    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:53.628840 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:53.628816    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:54.503654 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:54.503622    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:55.505186 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:55.505155    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:56.503503 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:56.503476    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:57.503867 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:57.503820    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:58.506244 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:58.506218    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:59.505042 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:59.505013    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:23:59.664640 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:59.664606    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:23:59.665537 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:59.665521    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:23:59.665621 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:59.665549    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:23:59.665621 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:23:59.665559    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:23:59.665726 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:59.665714    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:23:59.665886 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:59.665869    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:23:59.679146 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:23:59.679066    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:23:59.665824181 +0000 UTC m=+96.450524636,Count:5,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:00.297639 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:00.297605    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:00.504364 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:00.504340    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:00.629740 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:00.629685    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:00.630615 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:00.630588    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:00.630615 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:00.630621    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:00.630795 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:00.630631    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:00.630795 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:00.630658    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:00.653928 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:00.653905    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:01.505907 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:01.505878    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:02.504413 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:02.504381    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:03.506462 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:03.506431    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:03.569931 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:03.569901    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:24:04.504316 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:04.504287    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:05.504024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:05.503994    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:06.504472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:06.504440    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:07.306274 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:07.306232    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:07.503218 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:07.503190    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:07.654222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:07.654164    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:07.655104 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:07.655087    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:07.655172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:07.655118    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:07.655172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:07.655129    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:07.655172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:07.655155    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:07.684534 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:07.684509    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:08.504702 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:08.504674    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:09.506188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:09.506161    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:10.509960 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:10.509931    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:11.032134 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:11.032104    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 14:24:11.505528 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:11.505452    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:12.504437 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:12.504413    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:12.664570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:12.664539    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:12.665770 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:12.665749    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:12.665919 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:12.665779    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:12.665919 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:12.665789    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:12.666036 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:12.666014    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:12.666174 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:12.666156    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:24:12.677470 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:12.677391    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:24:12.666122845 +0000 UTC m=+109.450823299,Count:6,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:13.504760 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:13.504730    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:13.570331 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:13.570287    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:24:14.160546 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:14.160514    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 14:24:14.315644 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:14.315612    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:14.508393 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:14.508321    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:14.685510 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:14.685475    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:14.686405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:14.686390    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:14.686499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:14.686423    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:14.686499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:14.686436    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:14.686499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:14.686471    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:14.712825 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:14.712799    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:15.504585 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:15.504551    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:15.583285 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:15.583255    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:24:16.503417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:16.503388    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:17.504944 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:17.504917    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:18.504409 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:18.504377    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:19.504451 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:19.504421    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:20.503849 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:20.503803    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:21.325264 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:21.325230    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:21.502927 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:21.502897    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:21.713306 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:21.713232    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:21.714198 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:21.714180    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:21.714299 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:21.714215    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:21.714299 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:21.714229    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:21.714299 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:21.714259    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:21.735544 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:21.735519    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:22.502247 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:22.502218    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:23.505266 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:23.505236    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:23.571060 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:23.571026    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:24:23.665049 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:23.665027    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:23.666469 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:23.666456    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:23.666535 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:23.666483    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:23.666535 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:23.666495    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:23.666692 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:23.666681    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:23.666843 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:23.666814    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 40s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:24:23.676243 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:23.676161    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:24:23.666786065 +0000 UTC m=+120.451486526,Count:7,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:24.504194 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:24.504165    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:25.505021 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:25.504981    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:26.504972 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:26.504946    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:27.071759 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:27.071726    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 24 14:24:27.504191 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:27.504119    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:28.335615 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:28.335580    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:28.508975 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:28.508942    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:28.736595 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:28.736517    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:28.738799 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:28.738782    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:28.738915 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:28.738813    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:28.738915 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:28.738825    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:28.738915 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:28.738867    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:28.766748 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:28.766728    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:29.503346 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:29.503315    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:30.505385 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:30.505357    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:31.507152 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:31.507124    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:32.503583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:32.503559    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:33.506128 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:33.506092    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:33.571887 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:33.571859    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:24:34.504529 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.504502    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:34.664645 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.664612    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:34.665532 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.665516    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:34.665601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.665545    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:34.665601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.665554    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:34.665775 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:34.665763    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:34.675418 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:34.675322    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc87ffdf954  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:37.695629652 +0000 UTC m=+14.480330094,LastTimestamp:2026-07-24 14:24:34.6664457 +0000 UTC m=+131.451146162,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:34.745434 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:34.745366    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e90df963  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.573384035 +0000 UTC m=+3.358084490,LastTimestamp:2026-07-24 14:24:34.734649724 +0000 UTC m=+131.519350170,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:34.756416 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:34.755363    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fc5e98b925f  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:26.581615199 +0000 UTC m=+3.366315663,LastTimestamp:2026-07-24 14:24:34.742584159 +0000 UTC m=+131.527284617,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:34.867415 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.867392    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/3.log"
Jul 24 14:24:34.867522 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.867439    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5"}
Jul 24 14:24:34.867612 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.867598    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:34.868296 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.868279    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:34.868360 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.868309    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:34.868360 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:34.868320    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:34.868501 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:34.868488    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:35.347932 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:35.347902    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:35.506646 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:35.506619    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:35.767332 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:35.767313    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:35.768269 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:35.768255    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:35.768343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:35.768285    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:35.768343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:35.768299    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:35.768343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:35.768328    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:35.790430 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:35.790405    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:36.505458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:36.505432    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:37.503520 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:37.503491    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:38.504572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:38.504536    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:39.507167 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:39.507132    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:40.509001 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:40.508974    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:41.503140 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:41.503105    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:42.363251 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:42.363214    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:42.506063 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:42.506036    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:42.791405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:42.791386    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:42.792300 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:42.792283    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:42.792406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:42.792317    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:42.792406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:42.792331    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:42.792406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:42.792363    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:42.814098 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:42.814072    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:43.504976 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:43.504940    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:43.572200 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:43.572161    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:24:44.505985 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:44.505954    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:45.506452 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.506424    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:45.885476 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.885450    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/4.log"
Jul 24 14:24:45.885879 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.885865    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/3.log"
Jul 24 14:24:45.885941 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.885896    2584 generic.go:356] "Generic (PLEG): container finished" podID="74ca4cffd660659a68e7ccdd89a85ecc" containerID="a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5" exitCode=1
Jul 24 14:24:45.885941 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.885921    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerDied","Data":"a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5"}
Jul 24 14:24:45.886029 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.885962    2584 scope.go:117] "RemoveContainer" containerID="b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4"
Jul 24 14:24:45.886077 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.886063    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:45.887138 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.887123    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:45.887191 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.887154    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:45.887191 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.887164    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:45.887344 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:45.887332    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:45.887394 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:45.887383    2584 scope.go:117] "RemoveContainer" containerID="b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4"
Jul 24 14:24:45.895519 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:45.895484    2584 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_3 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4'" containerID="b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4"
Jul 24 14:24:45.895611 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:45.895530    2584 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_3 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'b3ad84f50be0687571f0b34bc9e0a9b0a7771753846feb53687ad6c9d3aa42b4'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" logger="UnhandledError"
Jul 24 14:24:45.895677 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:45.895657    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:24:45.914441 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:45.914370    2584 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal.18c53fcb4cd2b1f3  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal,UID:74ca4cffd660659a68e7ccdd89a85ecc,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc),Source:EventSource{Component:kubelet,Host:ip-10-0-135-93.ec2.internal,},FirstTimestamp:2026-07-24 14:22:49.722057203 +0000 UTC m=+26.506757661,LastTimestamp:2026-07-24 14:24:45.895624845 +0000 UTC m=+142.680325303,Count:8,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-93.ec2.internal,}"
Jul 24 14:24:46.504993 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:46.504965    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:46.888562 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:46.888538    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/4.log"
Jul 24 14:24:47.508759 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:47.508726    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:48.504570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:48.504538    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:49.372357 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:49.372322    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:49.506971 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:49.506943    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:49.814655 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:49.814620    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:49.815613 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:49.815589    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:49.815726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:49.815626    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:49.815726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:49.815637    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:49.815726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:49.815672    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:49.836615 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:49.836590    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:50.504957 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:50.504929    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:51.460131 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:51.460090    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:51.461080 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:51.461059    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:51.461211 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:51.461092    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:51.461211 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:51.461104    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:51.517157 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:51.517132    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:51.873236 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:51.873205    2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 14:24:52.503576 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:52.503549    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:52.664659 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:52.664622    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:52.665542 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:52.665521    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:52.665636 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:52.665564    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:52.665636 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:52.665578    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:52.665775 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:52.665762    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:53.503709 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:53.503677    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:53.572506 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:53.572482    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:24:54.506617 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:54.506586    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:55.504648 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:55.504613    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:56.381574 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:56.381538    2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 24 14:24:56.503927 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:56.503901    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:56.837193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:56.837161    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:24:56.838107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:56.838087    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:24:56.838227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:56.838125    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:24:56.838227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:56.838141    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:24:56.838227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:56.838176    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:56.861412 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:56.861385    2584 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:24:57.505267 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:57.505238    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:58.505971 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:58.505938    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 14:24:58.820342 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:58.820316    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-98vkc"
Jul 24 14:24:59.449731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.449693    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 14:24:59.523461 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.523435    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:24:59.546275 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.546253    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:24:59.614124 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.614105    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:24:59.824511 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.824481    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 14:19:58 +0000 UTC" deadline="2028-03-17 10:38:45.838855202 +0000 UTC"
Jul 24 14:24:59.824511 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.824509    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14444h13m46.014349633s"
Jul 24 14:24:59.889907 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.889891    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:24:59.889987 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:24:59.889912    2584 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:24:59.944297 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.944276    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:24:59.976963 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:24:59.976949    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.054665 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.054647    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.327695 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.327670    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.327695 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:00.327692    2584 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.585596 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.585535    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.610825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.610805    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.665246 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.665223    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:25:00.666119 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.666104    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:25:00.666192 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.666132    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:25:00.666192 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.666141    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:25:00.666355 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:00.666341    2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:25:00.666508 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:00.666489    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:25:00.677932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.677911    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.948455 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:00.948389    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:00.948455 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:00.948408    2584 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:02.925355 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:02.925326    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:03.027665 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.027642    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:03.090904 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.090883    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:03.365453 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.365430    2584 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:03.365533 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:03.365455    2584 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-135-93.ec2.internal" not found
Jul 24 14:25:03.399879 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:03.399823    2584 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-93.ec2.internal\" not found" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:25:03.424097 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.424074    2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:25:03.573115 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:03.573081    2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:25:03.861654 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.861628    2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 14:25:03.862685 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.862662    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 14:25:03.862786 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.862697    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 14:25:03.862786 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.862712    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeHasSufficientPID"
Jul 24 14:25:03.862786 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.862745    2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:25:03.875871 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:03.875856    2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-93.ec2.internal"
Jul 24 14:25:03.875966 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:03.875876    2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-93.ec2.internal\": node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:25:03.899078 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:03.899063    2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-93.ec2.internal\" not found"
Jul 24 14:25:03.999466 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:03.999424    2584 kubelet_node_status.go:448] "Node not becoming ready in time after startup"
Jul 24 14:25:08.594154 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:08.594118    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:09.437989 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.437966    2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:25:09.503183 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.503163    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal"
Jul 24 14:25:09.526127 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.526107    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 14:25:09.527501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.527489    2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal"
Jul 24 14:25:09.542771 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.542755    2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 14:25:09.574345 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.574328    2584 apiserver.go:52] "Watching apiserver"
Jul 24 14:25:09.584290 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.584271    2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 14:25:09.584621 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.584601    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-hpn4m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal","openshift-multus/multus-t899w","openshift-multus/network-metrics-daemon-4gdl8","kube-system/global-pull-secret-syncer-wz7zb","openshift-cluster-node-tuning-operator/tuned-bmcjl","openshift-multus/multus-additional-cni-plugins-54lzz","openshift-network-diagnostics/network-check-target-jfv8z","openshift-network-operator/iptables-alerter-4298x","openshift-ovn-kubernetes/ovnkube-node-f9csd","kube-system/konnectivity-agent-lmt58","kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b","openshift-dns/node-resolver-l2htf"]
Jul 24 14:25:09.587072 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.587054    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.589161 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.589141    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.591344 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.591329    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.592036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.592019    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lrqwn\""
Jul 24 14:25:09.592125 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.592019    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.592125 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.592025    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.593563 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.593546    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.595679 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.595659    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 14:25:09.596025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.595784    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:09.596025 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.595892    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:09.598137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.598125    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:09.598188 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.598173    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:09.600347 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.600330    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.602909 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.602474    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.602909 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.602710    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.603260 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603214    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 14:25:09.603356 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603261    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 14:25:09.603356 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603310    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dbl6n\""
Jul 24 14:25:09.603356 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603324    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 14:25:09.603518 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603265    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 14:25:09.603518 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603389    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.603518 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603476    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.603651 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603610    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.603651 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.603641    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4w69v\""
Jul 24 14:25:09.604367 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.604351    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.604447 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.604427    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:09.605027 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605008    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.605166 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605152    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 14:25:09.605242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605184    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 14:25:09.605242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605231    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 14:25:09.605242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605235    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.605967 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605949    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 14:25:09.606033 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.605949    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2bnq4\""
Jul 24 14:25:09.607029 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.607012    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.608589 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.608571    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-747zc\""
Jul 24 14:25:09.608589 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.608585    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.609166 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.609154    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.611501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.611482    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.613906 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.613886    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.614082 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.614061    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:25:09.620553 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.620515    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.620682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.620615    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.620682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.620645    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 14:25:09.620803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.620681    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 14:25:09.620803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.620731    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 14:25:09.621785 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.621767    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 14:25:09.621785 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.621783    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9295s\""
Jul 24 14:25:09.621962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.621814    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 14:25:09.621962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.621928    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2q5gx\""
Jul 24 14:25:09.622307 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.621996    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kwpgn\""
Jul 24 14:25:09.622307 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.622152    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 14:25:09.622307 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.622150    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jpbs5\""
Jul 24 14:25:09.622307 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.622214    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 14:25:09.631487 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631468    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-socket-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.631573 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631494    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be0981fc-a012-4c1d-8282-653973ba75b5-tmp-dir\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.631573 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631510    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-systemd\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.631573 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631527    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.631573 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631552    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-system-cni-dir\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631592    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d36989a-3602-4e05-8ab1-7f85407fd61c-kubelet-config\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631605    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysconfig\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631619    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-systemd\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631640    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-kubelet\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631669    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-systemd-units\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631696    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysctl-d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.631719 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631720    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysctl-conf\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631735    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-lib-modules\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631750    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-var-lib-kubelet\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631764    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631798    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-tmp\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631818    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631862    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-ovnkube-script-lib\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631884    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d36989a-3602-4e05-8ab1-7f85407fd61c-dbus\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631899    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-host\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631931    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/be0981fc-a012-4c1d-8282-653973ba75b5-hosts-file\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631949    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-etc-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631963    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.631979    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-os-release\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.632026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632013    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632057    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-registration-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632074    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-sys-fs\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632089    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvcfk\" (UniqueName: \"kubernetes.io/projected/be0981fc-a012-4c1d-8282-653973ba75b5-kube-api-access-gvcfk\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632106    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-cnibin\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-sys\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632164    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-slash\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632178    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-env-overrides\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632191    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632227    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbvw2\" (UniqueName: \"kubernetes.io/projected/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-kube-api-access-vbvw2\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632260    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-run-ovn-kubernetes\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632288    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632319    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llc82\" (UniqueName: \"kubernetes.io/projected/100842d2-32ef-4422-b1ef-039e00c95e3e-kube-api-access-llc82\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632345    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-kubernetes\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632382    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-node-log\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632427    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v478m\" (UniqueName: \"kubernetes.io/projected/0988fdd9-d130-4737-b679-c74766d01475-kube-api-access-v478m\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.632497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632450    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-modprobe-d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-cni-netd\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tw7d\" (UniqueName: \"kubernetes.io/projected/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-kube-api-access-9tw7d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632533    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-ovn\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632554    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-cni-bin\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632572    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632586    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632623    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-run-netns\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632657    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0988fdd9-d130-4737-b679-c74766d01475-ovn-node-metrics-cert\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632682    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-var-lib-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632708    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-log-socket\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632730    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-tuned\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632752    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-device-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632778    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-ovnkube-config\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632805    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frnfd\" (UniqueName: \"kubernetes.io/projected/7478e2db-f305-4c13-8f1b-2e881537d6a4-kube-api-access-frnfd\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:09.633122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.632828    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-run\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.640264 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.640246    2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 14:25:09.704010 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.703967    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 14:25:09.733565 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733542    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tw7d\" (UniqueName: \"kubernetes.io/projected/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-kube-api-access-9tw7d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.733643 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733567    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-ovn\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.733643 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-cni-bin\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.733643 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733600    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:09.733643 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.733859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733642    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-ovn\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.733859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733653    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-socket-dir-parent\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.733859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733658    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-cni-bin\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.733859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733693    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-k8s-cni-cncf-io\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.733859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733818    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-netns\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733873    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-run-netns\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733898    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0988fdd9-d130-4737-b679-c74766d01475-ovn-node-metrics-cert\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733922    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/366e9dfd-b78f-4684-8649-cb33cfdd0e4f-agent-certs\") pod \"konnectivity-agent-lmt58\" (UID: \"366e9dfd-b78f-4684-8649-cb33cfdd0e4f\") " pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733945    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-system-cni-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733970    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-conf-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.733973    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-run-netns\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734000    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdzt9\" (UniqueName: \"kubernetes.io/projected/f6946e53-85a2-496e-ae5d-5becc6684eaa-kube-api-access-pdzt9\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734019    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjqw\" (UniqueName: \"kubernetes.io/projected/736d187e-5b4a-41b9-a918-a723cec1dcd7-kube-api-access-hnjqw\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734046    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-var-lib-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734067    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-log-socket\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734088    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a19cc3c-28fb-4aaf-adb8-8716fe896955-host-slash\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734107    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-kubelet\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734108    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-var-lib-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-tuned\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734147    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-log-socket\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734149    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-device-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734189    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-ovnkube-config\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734194    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-device-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734204    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734215    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frnfd\" (UniqueName: \"kubernetes.io/projected/7478e2db-f305-4c13-8f1b-2e881537d6a4-kube-api-access-frnfd\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6946e53-85a2-496e-ae5d-5becc6684eaa-cni-binary-copy\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734270    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-etc-kubernetes\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-run\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734314    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-socket-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734337    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be0981fc-a012-4c1d-8282-653973ba75b5-tmp-dir\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734376    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-systemd\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734379    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-run\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.734753 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734403    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734426    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-systemd\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734430    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-system-cni-dir\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734456    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d36989a-3602-4e05-8ab1-7f85407fd61c-kubelet-config\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysconfig\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734530    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-system-cni-dir\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734539    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-systemd\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734578    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-systemd\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-kubelet\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734589    2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-socket-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734610    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-systemd-units\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734611    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d36989a-3602-4e05-8ab1-7f85407fd61c-kubelet-config\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734620    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysconfig\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734623    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be0981fc-a012-4c1d-8282-653973ba75b5-tmp-dir\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.735497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734646    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-kubelet\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734654    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysctl-d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734659    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-systemd-units\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysctl-conf\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734702    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-lib-modules\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734723    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysctl-d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734738    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-ovnkube-config\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-var-lib-kubelet\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734777    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734786    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-var-lib-kubelet\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-daemon-config\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734808    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-lib-modules\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734853    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/736d187e-5b4a-41b9-a918-a723cec1dcd7-serviceca\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734877    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-tmp\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734889    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.734902    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734854    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-sysctl-conf\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734914    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.736220 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-run-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.734951    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-ovnkube-script-lib\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.734976    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:25:10.23495759 +0000 UTC m=+167.019658032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735031    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d36989a-3602-4e05-8ab1-7f85407fd61c-dbus\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735055    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-host\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735076    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/be0981fc-a012-4c1d-8282-653973ba75b5-hosts-file\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735096    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-etc-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735116    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735135    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-os-release\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735145    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-host\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735150    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d36989a-3602-4e05-8ab1-7f85407fd61c-dbus\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735158    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a19cc3c-28fb-4aaf-adb8-8716fe896955-iptables-alerter-script\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735195    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-etc-openvswitch\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735213    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj4d5\" (UniqueName: \"kubernetes.io/projected/0a19cc3c-28fb-4aaf-adb8-8716fe896955-kube-api-access-gj4d5\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.735223    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735239    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/366e9dfd-b78f-4684-8649-cb33cfdd0e4f-konnectivity-ca\") pod \"konnectivity-agent-lmt58\" (UID: \"366e9dfd-b78f-4684-8649-cb33cfdd0e4f\") " pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.737054 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735248    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/be0981fc-a012-4c1d-8282-653973ba75b5-hosts-file\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.735263    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:10.235248367 +0000 UTC m=+167.019948823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735296    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735307    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-os-release\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735318    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-cni-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735336    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-os-release\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735356    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-cni-bin\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735373    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-hostroot\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735389    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735395    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-registration-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735416    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-sys-fs\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735436    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvcfk\" (UniqueName: \"kubernetes.io/projected/be0981fc-a012-4c1d-8282-653973ba75b5-kube-api-access-gvcfk\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735443    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-ovnkube-script-lib\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735444    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-registration-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735475    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-cnibin\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735492    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-sys-fs\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.737912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-cni-multus\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735527    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-sys\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735551    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-slash\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735556    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-cnibin\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735573    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-env-overrides\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735633    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-slash\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735672    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-multus-certs\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735688    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-sys\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735716    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735734    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbvw2\" (UniqueName: \"kubernetes.io/projected/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-kube-api-access-vbvw2\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735755    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-run-ovn-kubernetes\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735773    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735811    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735848    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llc82\" (UniqueName: \"kubernetes.io/projected/100842d2-32ef-4422-b1ef-039e00c95e3e-kube-api-access-llc82\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735958    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-kubernetes\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.735963    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0988fdd9-d130-4737-b679-c74766d01475-env-overrides\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736029    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-kubernetes\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736057    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/100842d2-32ef-4422-b1ef-039e00c95e3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736063    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-node-log\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736092    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v478m\" (UniqueName: \"kubernetes.io/projected/0988fdd9-d130-4737-b679-c74766d01475-kube-api-access-v478m\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736128    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736135    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-node-log\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736101    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-run-ovn-kubernetes\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736153    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/736d187e-5b4a-41b9-a918-a723cec1dcd7-host\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736180    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-modprobe-d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-cni-netd\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736227    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-cnibin\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736314    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0988fdd9-d130-4737-b679-c74766d01475-host-cni-netd\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736333    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-modprobe-d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.736539    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/100842d2-32ef-4422-b1ef-039e00c95e3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.738167    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-tmp\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.738256    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-etc-tuned\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.738937 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.738355    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0988fdd9-d130-4737-b679-c74766d01475-ovn-node-metrics-cert\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.778060 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.778037    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvcfk\" (UniqueName: \"kubernetes.io/projected/be0981fc-a012-4c1d-8282-653973ba75b5-kube-api-access-gvcfk\") pod \"node-resolver-l2htf\" (UID: \"be0981fc-a012-4c1d-8282-653973ba75b5\") " pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.778454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.778430    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6mqjd"
Jul 24 14:25:09.778867 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.778850    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v478m\" (UniqueName: \"kubernetes.io/projected/0988fdd9-d130-4737-b679-c74766d01475-kube-api-access-v478m\") pod \"ovnkube-node-f9csd\" (UID: \"0988fdd9-d130-4737-b679-c74766d01475\") " pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.779826 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.779809    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbvw2\" (UniqueName: \"kubernetes.io/projected/1db402e3-9c22-4a23-9ba0-4b7e6b7eccad-kube-api-access-vbvw2\") pod \"aws-ebs-csi-driver-node-bzx4b\" (UID: \"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.788844 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.788815    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llc82\" (UniqueName: \"kubernetes.io/projected/100842d2-32ef-4422-b1ef-039e00c95e3e-kube-api-access-llc82\") pod \"multus-additional-cni-plugins-54lzz\" (UID: \"100842d2-32ef-4422-b1ef-039e00c95e3e\") " pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.800727 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.800712    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:09.800796 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.800731    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:09.800796 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.800741    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:09.800796 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:09.800792    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:10.30077167 +0000 UTC m=+167.085472130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:09.804188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.803588    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frnfd\" (UniqueName: \"kubernetes.io/projected/7478e2db-f305-4c13-8f1b-2e881537d6a4-kube-api-access-frnfd\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:09.804188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.803920    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tw7d\" (UniqueName: \"kubernetes.io/projected/07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd-kube-api-access-9tw7d\") pod \"tuned-bmcjl\" (UID: \"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd\") " pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.810296 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.810278    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6mqjd"
Jul 24 14:25:09.836480 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836457    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-daemon-config\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836540 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836485    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/736d187e-5b4a-41b9-a918-a723cec1dcd7-serviceca\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.836540 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836511    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a19cc3c-28fb-4aaf-adb8-8716fe896955-iptables-alerter-script\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.836540 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836534    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj4d5\" (UniqueName: \"kubernetes.io/projected/0a19cc3c-28fb-4aaf-adb8-8716fe896955-kube-api-access-gj4d5\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.836646 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836556    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/366e9dfd-b78f-4684-8649-cb33cfdd0e4f-konnectivity-ca\") pod \"konnectivity-agent-lmt58\" (UID: \"366e9dfd-b78f-4684-8649-cb33cfdd0e4f\") " pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.836706 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-cni-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836748 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-os-release\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836796 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836758    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-cni-bin\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836796 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836781    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-hostroot\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836809    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-cni-multus\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836807    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-cni-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836851    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-multus-certs\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836883    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/736d187e-5b4a-41b9-a918-a723cec1dcd7-host\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836885    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-hostroot\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836887    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-cni-multus\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836910    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-cnibin\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.836922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836919    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-multus-certs\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836939    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-cni-bin\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836848    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-os-release\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836970    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-cnibin\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836960    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-socket-dir-parent\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.836990    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/736d187e-5b4a-41b9-a918-a723cec1dcd7-host\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-k8s-cni-cncf-io\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-socket-dir-parent\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837027    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-netns\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837029    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/736d187e-5b4a-41b9-a918-a723cec1dcd7-serviceca\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837050    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-k8s-cni-cncf-io\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837052    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/366e9dfd-b78f-4684-8649-cb33cfdd0e4f-agent-certs\") pod \"konnectivity-agent-lmt58\" (UID: \"366e9dfd-b78f-4684-8649-cb33cfdd0e4f\") " pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-system-cni-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837090    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-run-netns\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837095    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/366e9dfd-b78f-4684-8649-cb33cfdd0e4f-konnectivity-ca\") pod \"konnectivity-agent-lmt58\" (UID: \"366e9dfd-b78f-4684-8649-cb33cfdd0e4f\") " pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837098    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-conf-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837117    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0a19cc3c-28fb-4aaf-adb8-8716fe896955-iptables-alerter-script\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837127    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdzt9\" (UniqueName: \"kubernetes.io/projected/f6946e53-85a2-496e-ae5d-5becc6684eaa-kube-api-access-pdzt9\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837130    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-conf-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837129    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-system-cni-dir\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837332    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjqw\" (UniqueName: \"kubernetes.io/projected/736d187e-5b4a-41b9-a918-a723cec1dcd7-kube-api-access-hnjqw\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837350    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a19cc3c-28fb-4aaf-adb8-8716fe896955-host-slash\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-kubelet\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837396    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6946e53-85a2-496e-ae5d-5becc6684eaa-cni-binary-copy\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837410    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-etc-kubernetes\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837415    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0a19cc3c-28fb-4aaf-adb8-8716fe896955-host-slash\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837452    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-host-var-lib-kubelet\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837468    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6946e53-85a2-496e-ae5d-5becc6684eaa-etc-kubernetes\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.837731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837602    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6946e53-85a2-496e-ae5d-5becc6684eaa-multus-daemon-config\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.838021 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.837770    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6946e53-85a2-496e-ae5d-5becc6684eaa-cni-binary-copy\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.838976 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.838959    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/366e9dfd-b78f-4684-8649-cb33cfdd0e4f-agent-certs\") pod \"konnectivity-agent-lmt58\" (UID: \"366e9dfd-b78f-4684-8649-cb33cfdd0e4f\") " pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.858208 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.858187    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjqw\" (UniqueName: \"kubernetes.io/projected/736d187e-5b4a-41b9-a918-a723cec1dcd7-kube-api-access-hnjqw\") pod \"node-ca-hpn4m\" (UID: \"736d187e-5b4a-41b9-a918-a723cec1dcd7\") " pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.861714 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.861698    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdzt9\" (UniqueName: \"kubernetes.io/projected/f6946e53-85a2-496e-ae5d-5becc6684eaa-kube-api-access-pdzt9\") pod \"multus-t899w\" (UID: \"f6946e53-85a2-496e-ae5d-5becc6684eaa\") " pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.872869 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.872848    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj4d5\" (UniqueName: \"kubernetes.io/projected/0a19cc3c-28fb-4aaf-adb8-8716fe896955-kube-api-access-gj4d5\") pod \"iptables-alerter-4298x\" (UID: \"0a19cc3c-28fb-4aaf-adb8-8716fe896955\") " pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.897083 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.897066    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bmcjl"
Jul 24 14:25:09.901711 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.901699    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2htf"
Jul 24 14:25:09.907170 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.907151    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b3a2ee_67dd_4810_9d4f_b80fc3a2bbdd.slice/crio-210df9675d8e2896bb20ebff73e63f3481fc3f3714d62db6d97e318ef65887b2 WatchSource:0}: Error finding container 210df9675d8e2896bb20ebff73e63f3481fc3f3714d62db6d97e318ef65887b2: Status 404 returned error can't find the container with id 210df9675d8e2896bb20ebff73e63f3481fc3f3714d62db6d97e318ef65887b2
Jul 24 14:25:09.908523 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.908489    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b"
Jul 24 14:25:09.911749 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.911730    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0981fc_a012_4c1d_8282_653973ba75b5.slice/crio-e7eb2a5dac3a06a0a7036397a6c86c66512fcb8164f8c6147620b703230298ca WatchSource:0}: Error finding container e7eb2a5dac3a06a0a7036397a6c86c66512fcb8164f8c6147620b703230298ca: Status 404 returned error can't find the container with id e7eb2a5dac3a06a0a7036397a6c86c66512fcb8164f8c6147620b703230298ca
Jul 24 14:25:09.913726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.913717    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:09.916549 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.916483    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db402e3_9c22_4a23_9ba0_4b7e6b7eccad.slice/crio-71421455b6650ccc2b06590a8c141249111714ccadd3d4628c66cd13c2b67b2c WatchSource:0}: Error finding container 71421455b6650ccc2b06590a8c141249111714ccadd3d4628c66cd13c2b67b2c: Status 404 returned error can't find the container with id 71421455b6650ccc2b06590a8c141249111714ccadd3d4628c66cd13c2b67b2c
Jul 24 14:25:09.919041 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.919014    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-54lzz"
Jul 24 14:25:09.922498 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.922478    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0988fdd9_d130_4737_b679_c74766d01475.slice/crio-62058bffa2141a5b1af772dff99e3d41d97bb95bb9f9e9a49b3bfe207770d14e WatchSource:0}: Error finding container 62058bffa2141a5b1af772dff99e3d41d97bb95bb9f9e9a49b3bfe207770d14e: Status 404 returned error can't find the container with id 62058bffa2141a5b1af772dff99e3d41d97bb95bb9f9e9a49b3bfe207770d14e
Jul 24 14:25:09.923171 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.923149    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b" event={"ID":"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad","Type":"ContainerStarted","Data":"71421455b6650ccc2b06590a8c141249111714ccadd3d4628c66cd13c2b67b2c"}
Jul 24 14:25:09.924295 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.924260    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2htf" event={"ID":"be0981fc-a012-4c1d-8282-653973ba75b5","Type":"ContainerStarted","Data":"e7eb2a5dac3a06a0a7036397a6c86c66512fcb8164f8c6147620b703230298ca"}
Jul 24 14:25:09.924998 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.924984    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4298x"
Jul 24 14:25:09.925394 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.925371    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bmcjl" event={"ID":"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd","Type":"ContainerStarted","Data":"210df9675d8e2896bb20ebff73e63f3481fc3f3714d62db6d97e318ef65887b2"}
Jul 24 14:25:09.927734 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.927715    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100842d2_32ef_4422_b1ef_039e00c95e3e.slice/crio-11914f66b0c727f8e4a656d07deafe7b7ab1e5fb56e88655cfa1935ae6f380e8 WatchSource:0}: Error finding container 11914f66b0c727f8e4a656d07deafe7b7ab1e5fb56e88655cfa1935ae6f380e8: Status 404 returned error can't find the container with id 11914f66b0c727f8e4a656d07deafe7b7ab1e5fb56e88655cfa1935ae6f380e8
Jul 24 14:25:09.931038 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.931018    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hpn4m"
Jul 24 14:25:09.932985 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.932965    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a19cc3c_28fb_4aaf_adb8_8716fe896955.slice/crio-b1f85e3ebfcb4c01fbf1cb09c69b30f901a079a6ca7d36995f79aa8a6b5ad2f6 WatchSource:0}: Error finding container b1f85e3ebfcb4c01fbf1cb09c69b30f901a079a6ca7d36995f79aa8a6b5ad2f6: Status 404 returned error can't find the container with id b1f85e3ebfcb4c01fbf1cb09c69b30f901a079a6ca7d36995f79aa8a6b5ad2f6
Jul 24 14:25:09.935899 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.935881    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:09.938760 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.938744    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736d187e_5b4a_41b9_a918_a723cec1dcd7.slice/crio-bae7ddb499cea0ec58d95ed8aa21a205d0405d9bc18dcbaf152c5f18643557f7 WatchSource:0}: Error finding container bae7ddb499cea0ec58d95ed8aa21a205d0405d9bc18dcbaf152c5f18643557f7: Status 404 returned error can't find the container with id bae7ddb499cea0ec58d95ed8aa21a205d0405d9bc18dcbaf152c5f18643557f7
Jul 24 14:25:09.939579 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:09.939560    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t899w"
Jul 24 14:25:09.945586 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.945567    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366e9dfd_b78f_4684_8649_cb33cfdd0e4f.slice/crio-c26ca1aeaa93401ec2e94136e854cf38213d5ca976ff50a320677711d083e903 WatchSource:0}: Error finding container c26ca1aeaa93401ec2e94136e854cf38213d5ca976ff50a320677711d083e903: Status 404 returned error can't find the container with id c26ca1aeaa93401ec2e94136e854cf38213d5ca976ff50a320677711d083e903
Jul 24 14:25:09.949216 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:09.949197    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6946e53_85a2_496e_ae5d_5becc6684eaa.slice/crio-fb9e459b9ecdd62b7e2e01e5077e29d76992c97b87343a3881667afbfbf0b7ae WatchSource:0}: Error finding container fb9e459b9ecdd62b7e2e01e5077e29d76992c97b87343a3881667afbfbf0b7ae: Status 404 returned error can't find the container with id fb9e459b9ecdd62b7e2e01e5077e29d76992c97b87343a3881667afbfbf0b7ae
Jul 24 14:25:10.073373 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.073317    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-93.ec2.internal" podStartSLOduration=1.073301697 podStartE2EDuration="1.073301697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:10.046533019 +0000 UTC m=+166.831233483" watchObservedRunningTime="2026-07-24 14:25:10.073301697 +0000 UTC m=+166.858002159"
Jul 24 14:25:10.240677 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.240640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:10.240677 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.240677    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:10.240883 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.240783    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:10.240883 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.240784    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:10.240883 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.240846    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:11.240819914 +0000 UTC m=+168.025520357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:10.240883 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.240867    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:25:11.240852579 +0000 UTC m=+168.025553021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:10.341954 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.341872    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:10.342111 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.342066    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:10.342111 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.342091    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:10.342111 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.342106    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:10.342263 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.342175    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:11.342155148 +0000 UTC m=+168.126855607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:10.665155 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.665079    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:10.665812 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:10.665218    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:10.814746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.814700    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:20:09 +0000 UTC" deadline="2028-02-16 12:42:23.053001421 +0000 UTC"
Jul 24 14:25:10.814746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.814746    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13726h17m12.238260626s"
Jul 24 14:25:10.944729 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.944646    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hpn4m" event={"ID":"736d187e-5b4a-41b9-a918-a723cec1dcd7","Type":"ContainerStarted","Data":"bae7ddb499cea0ec58d95ed8aa21a205d0405d9bc18dcbaf152c5f18643557f7"}
Jul 24 14:25:10.949881 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.949847    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4298x" event={"ID":"0a19cc3c-28fb-4aaf-adb8-8716fe896955","Type":"ContainerStarted","Data":"b1f85e3ebfcb4c01fbf1cb09c69b30f901a079a6ca7d36995f79aa8a6b5ad2f6"}
Jul 24 14:25:10.958901 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.958243    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"62058bffa2141a5b1af772dff99e3d41d97bb95bb9f9e9a49b3bfe207770d14e"}
Jul 24 14:25:10.976610 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.976581    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t899w" event={"ID":"f6946e53-85a2-496e-ae5d-5becc6684eaa","Type":"ContainerStarted","Data":"fb9e459b9ecdd62b7e2e01e5077e29d76992c97b87343a3881667afbfbf0b7ae"}
Jul 24 14:25:10.992494 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.992417    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmt58" event={"ID":"366e9dfd-b78f-4684-8649-cb33cfdd0e4f","Type":"ContainerStarted","Data":"c26ca1aeaa93401ec2e94136e854cf38213d5ca976ff50a320677711d083e903"}
Jul 24 14:25:10.997068 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:10.997042    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerStarted","Data":"11914f66b0c727f8e4a656d07deafe7b7ab1e5fb56e88655cfa1935ae6f380e8"}
Jul 24 14:25:11.249073 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.248986    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:11.249073 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.249042    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:11.249294 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.249176    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:11.249294 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.249239    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:13.249220816 +0000 UTC m=+170.033921284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:11.249634 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.249611    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:11.249724 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.249669    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:25:13.249653237 +0000 UTC m=+170.034353696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:11.349768 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.349734    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:11.349969 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.349952    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:11.350030 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.349980    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:11.350276 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.350254    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:11.350487 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.350472    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:13.350447424 +0000 UTC m=+170.135147928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:11.666998 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.666965    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:11.667491 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.667093    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:11.667491 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.667168    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:11.667491 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:11.667239    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:11.815478 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.815434    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:20:09 +0000 UTC" deadline="2028-01-21 12:56:00.633859842 +0000 UTC"
Jul 24 14:25:11.815478 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:11.815475    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13102h30m48.818389735s"
Jul 24 14:25:12.664946 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:12.664589    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:12.664946 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:12.664726    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:13.266109 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:13.266072    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:13.266680 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:13.266130    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:13.266680 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.266289    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:13.266680 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.266350    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:17.266331268 +0000 UTC m=+174.051031712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:13.266680 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.266573    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:13.266680 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.266652    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:25:17.266633647 +0000 UTC m=+174.051334091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:13.351564 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:13.351520    2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:25:13.367286 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:13.367253    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:13.367487 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.367463    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:13.367585 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.367493    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:13.367585 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.367509    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:13.367585 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.367583    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:17.367563857 +0000 UTC m=+174.152264321 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:13.595542 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.595504    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:13.668094 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:13.668064    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:13.668271 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.668186    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:13.668271 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:13.668244    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:13.668390 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:13.668322    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:14.665686 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:14.665179    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:14.665686 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:14.665323    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:15.665591 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:15.664763    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:15.665591 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:15.664915    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:15.665591 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:15.664763    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:15.665591 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:15.665331    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:16.664499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:16.664464    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:16.664697 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:16.664609    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:17.298310 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:17.298207    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:17.298310 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:17.298260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:17.298823 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.298355    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:17.298823 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.298418    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:25:25.29840364 +0000 UTC m=+182.083104082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:17.298956 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.298356    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:17.299012 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.298970    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:25.298949469 +0000 UTC m=+182.083649913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:17.399422 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:17.399379    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:17.399586 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.399563    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:17.399647 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.399589    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:17.399647 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.399603    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:17.399767 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.399662    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:25.399644003 +0000 UTC m=+182.184344448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:17.665317 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:17.665240    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:17.665466 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:17.665254    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:17.665466 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.665382    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:17.665466 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:17.665454    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:18.596922 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:18.596869    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:18.665078 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:18.664925    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:18.665244 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:18.665099    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:19.664541 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:19.664507    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:19.665023 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:19.664656    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:19.665023 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:19.664664    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:19.665023 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:19.664758    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:20.664956 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:20.664911    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:20.665379 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:20.665059    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:21.664589 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:21.664556    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:21.664772 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:21.664764    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:21.664852 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:21.664789    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:21.664912 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:21.664861    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:21.665019 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:21.665000    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:25:22.664861 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:22.664814    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:22.665017 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:22.664952    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:23.597319 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:23.597267    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:23.665856 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:23.665367    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:23.665856 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:23.665474    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:23.665856 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:23.665536    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:23.665856 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:23.665648    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:24.665270 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:24.665237    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:24.665442 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:24.665400    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:25.352968 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:25.352931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:25.353422 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:25.353030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:25.353422 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.353104    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:25.353422 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.353132    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:25.353422 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.353183    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:41.353165467 +0000 UTC m=+198.137865913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:25.353422 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.353198    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:25:41.353192036 +0000 UTC m=+198.137892477 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:25.454105 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:25.454072    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:25.454280 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.454263    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:25.454339 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.454285    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:25.454339 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.454298    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:25.454428 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.454353    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:41.454338221 +0000 UTC m=+198.239038664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:25.665373 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:25.665292    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:25.665523 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:25.665292    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:25.665523 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.665434    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:25.665523 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:25.665500    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:26.664325 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:26.664289    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:26.664825 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:26.664418    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:27.152208 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:27.152180    2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 14:25:27.664323 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:27.664276    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:27.664406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:27.664276    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:27.664406 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:27.664388    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:27.664700 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:27.664446    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:28.039244 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.039027    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b" event={"ID":"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad","Type":"ContainerStarted","Data":"fa5032984050774cd2ef742ee3ab9a45ce96f4453ea549432345bf6e136b6de3"}
Jul 24 14:25:28.040983 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.040929    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t899w" event={"ID":"f6946e53-85a2-496e-ae5d-5becc6684eaa","Type":"ContainerStarted","Data":"556dd16a0169d058c16979a9d77d6b81b9664da6ccd58dbb237a9682a638b5da"}
Jul 24 14:25:28.043878 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.042544    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmt58" event={"ID":"366e9dfd-b78f-4684-8649-cb33cfdd0e4f","Type":"ContainerStarted","Data":"c71a65412855a24ee0d99ddef5e0200cf1095c513986acca84cdbaf8c96709b9"}
Jul 24 14:25:28.044990 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.044959    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerStarted","Data":"3d6de7b2806117f6449aaa8dcc4ddd74309270e01d9cf18f0753104dda6e6e9d"}
Jul 24 14:25:28.046871 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.046815    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2htf" event={"ID":"be0981fc-a012-4c1d-8282-653973ba75b5","Type":"ContainerStarted","Data":"d4c22ebef2051ceaf6378b942bc3b28c4b78141f15c1dfd5677950997526c18c"}
Jul 24 14:25:28.049075 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.048967    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bmcjl" event={"ID":"07b3a2ee-67dd-4810-9d4f-b80fc3a2bbdd","Type":"ContainerStarted","Data":"18389cc1100de4cd59f928d05609cfd94110783fd9626fafac37c18b3bd72258"}
Jul 24 14:25:28.051022 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.050999    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hpn4m" event={"ID":"736d187e-5b4a-41b9-a918-a723cec1dcd7","Type":"ContainerStarted","Data":"5b45ac73dd582a8b32196e4dd66b8107838cf848a469dc015cd4714b6d7b8a21"}
Jul 24 14:25:28.053377 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.053352    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"d43fef8454612ed203385211b7dc1d79259419297c954e80aa10fe7fd6547417"}
Jul 24 14:25:28.119591 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.119414    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t899w" podStartSLOduration=6.260955351 podStartE2EDuration="24.119400449s" totalImagesPullingTime="17.858445098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:28.119323054 +0000 UTC m=+184.904023519" watchObservedRunningTime="2026-07-24 14:25:28.119400449 +0000 UTC m=+184.904100913"
Jul 24 14:25:28.225634 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.225592    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lmt58" podStartSLOduration=6.770494022 podStartE2EDuration="24.225576675s" totalImagesPullingTime="17.455082653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:28.196004127 +0000 UTC m=+184.980704605" watchObservedRunningTime="2026-07-24 14:25:28.225576675 +0000 UTC m=+185.010277139"
Jul 24 14:25:28.255179 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.255141    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bmcjl" podStartSLOduration=7.760710983 podStartE2EDuration="25.255126494s" totalImagesPullingTime="17.494415511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:28.225982707 +0000 UTC m=+185.010683171" watchObservedRunningTime="2026-07-24 14:25:28.255126494 +0000 UTC m=+185.039826959"
Jul 24 14:25:28.299974 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.299930    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l2htf" podStartSLOduration=7.404565653 podStartE2EDuration="25.299913508s" totalImagesPullingTime="17.895347855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:28.255497539 +0000 UTC m=+185.040198002" watchObservedRunningTime="2026-07-24 14:25:28.299913508 +0000 UTC m=+185.084614026"
Jul 24 14:25:28.300106 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.299998    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hpn4m" podStartSLOduration=6.457182712 podStartE2EDuration="24.299994361s" totalImagesPullingTime="17.842811649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:28.299491123 +0000 UTC m=+185.084191589" watchObservedRunningTime="2026-07-24 14:25:28.299994361 +0000 UTC m=+185.084694824"
Jul 24 14:25:28.598361 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:28.598321    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:28.664862 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:28.664818    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:28.665374 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:28.664962    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:29.040103 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.040085    2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 14:25:29.058072 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.058044    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"652cc57e14721a870197ccef6cbd4dfd29bb1350a9de9e8113a9255172e1f505"}
Jul 24 14:25:29.058182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.058122    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"0bc52db43fb204cebc0eeb6dcaa347eac401918092811a874fdc431633f3570f"}
Jul 24 14:25:29.058182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.058152    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"85cdabae5da8f2b66fa76f39302847af8ffcf78d6962251809b1b2fb96f6a84d"}
Jul 24 14:25:29.058278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.058198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"ff6a66d1d9018cc9bfcd074565cea32044e3a848349701a1cea6cf0ee56ed203"}
Jul 24 14:25:29.058278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.058226    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"4538033519de10647d0505da9df179bf5699dc61cd34e304bd533c392bf27768"}
Jul 24 14:25:29.059558 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.059534    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b" event={"ID":"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad","Type":"ContainerStarted","Data":"8ffd8d3563b57649405eca545726a25c4ee7de801df06d63cdaf97d846b5fa8d"}
Jul 24 14:25:29.060777 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.060758    2584 generic.go:356] "Generic (PLEG): container finished" podID="100842d2-32ef-4422-b1ef-039e00c95e3e" containerID="3d6de7b2806117f6449aaa8dcc4ddd74309270e01d9cf18f0753104dda6e6e9d" exitCode=0
Jul 24 14:25:29.060885 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.060864    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerDied","Data":"3d6de7b2806117f6449aaa8dcc4ddd74309270e01d9cf18f0753104dda6e6e9d"}
Jul 24 14:25:29.665035 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.665000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:29.665035 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.665036    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:29.665618 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:29.665124    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:29.665618 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:29.665204    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:29.667188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.667105    2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T14:25:29.040099995Z","UUID":"5ef733b9-2cd1-43be-bf6c-d67f25fa2032","Handler":null,"Name":"","Endpoint":""}
Jul 24 14:25:29.669294 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.669265    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 14:25:29.669423 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.669314    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 14:25:29.936594 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.936536    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:29.937186 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:29.937168    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:30.065530 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.065497    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b" event={"ID":"1db402e3-9c22-4a23-9ba0-4b7e6b7eccad","Type":"ContainerStarted","Data":"7f4440921203608b12cb972cc956ed80f603999ca6b612a8691f2c7b5e4a0475"}
Jul 24 14:25:30.067111 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.067084    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4298x" event={"ID":"0a19cc3c-28fb-4aaf-adb8-8716fe896955","Type":"ContainerStarted","Data":"cea8419e579e8ad9a560d617859468d06d80c4ca10bf811316fd281e5cd81e17"}
Jul 24 14:25:30.067242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.067167    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:30.067686 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.067667    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lmt58"
Jul 24 14:25:30.125786 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.125741    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzx4b" podStartSLOduration=7.901168259 podStartE2EDuration="27.125727436s" totalImagesPullingTime="19.224559177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:30.10107931 +0000 UTC m=+186.885779774" watchObservedRunningTime="2026-07-24 14:25:30.125727436 +0000 UTC m=+186.910427900"
Jul 24 14:25:30.168713 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.168666    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4298x" podStartSLOduration=8.291040659 podStartE2EDuration="26.16865133s" totalImagesPullingTime="17.877610671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:30.128827643 +0000 UTC m=+186.913528107" watchObservedRunningTime="2026-07-24 14:25:30.16865133 +0000 UTC m=+186.953351791"
Jul 24 14:25:30.664500 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:30.664415    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:30.664655 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:30.664556    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:31.072722 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:31.072688    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"c547cc9b80c7a13c53fccf68db8cadb3ab9742de519e4ca124ceebc3eabf02a0"}
Jul 24 14:25:31.664627 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:31.664593    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:31.664795 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:31.664607    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:31.664795 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:31.664703    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:31.664923 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:31.664810    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:32.665328 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:32.665096    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:32.665873 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:32.665455    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:33.080814 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.080771    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" event={"ID":"0988fdd9-d130-4737-b679-c74766d01475","Type":"ContainerStarted","Data":"e454277d180bef7daf963bebe38e3978e2cc87baf71e50b60c034df7790476ab"}
Jul 24 14:25:33.081146 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.081125    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:33.081265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.081204    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:33.097888 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.097863    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:33.135607 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.135552    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd" podStartSLOduration=12.199216171 podStartE2EDuration="30.135532488s" totalImagesPullingTime="17.936316317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:33.135304147 +0000 UTC m=+189.920004613" watchObservedRunningTime="2026-07-24 14:25:33.135532488 +0000 UTC m=+189.920232952"
Jul 24 14:25:33.598943 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:33.598912    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:33.665640 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.665610    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:33.666121 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:33.665712    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:33.666180 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:33.666134    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:33.666265 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:33.666239    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:33.666409 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:33.666285    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:25:34.083605 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:34.083575    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:34.097698 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:34.097672    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:25:34.665265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:34.665087    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:34.665459 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:34.665350    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:34.995849 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:34.995806    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wz7zb"]
Jul 24 14:25:34.996226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:34.995927    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:34.996226 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:34.996008    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:35.001561 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:35.001539    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4gdl8"]
Jul 24 14:25:35.009595 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:35.009573    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jfv8z"]
Jul 24 14:25:35.009688 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:35.009667    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:35.009773 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:35.009756    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:35.086816 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:35.086788    2584 generic.go:356] "Generic (PLEG): container finished" podID="100842d2-32ef-4422-b1ef-039e00c95e3e" containerID="deb09c11b0592e673b02239b5ba418bf3d0383b10bb154bb6e21f6e439af006e" exitCode=0
Jul 24 14:25:35.086963 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:35.086883    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerDied","Data":"deb09c11b0592e673b02239b5ba418bf3d0383b10bb154bb6e21f6e439af006e"}
Jul 24 14:25:35.086963 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:35.086919    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:35.087398 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:35.087362    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:36.665027 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:36.665007    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:36.665350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:36.665102    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:36.665350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:36.665110    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:36.665350 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:36.665108    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:36.665350 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:36.665183    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:36.665350 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:36.665225    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:37.093708 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:37.093679    2584 generic.go:356] "Generic (PLEG): container finished" podID="100842d2-32ef-4422-b1ef-039e00c95e3e" containerID="0f973a9e7bd03af34086019791992f2909d9d0efd432abdd67b0115618387626" exitCode=0
Jul 24 14:25:37.093893 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:37.093721    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerDied","Data":"0f973a9e7bd03af34086019791992f2909d9d0efd432abdd67b0115618387626"}
Jul 24 14:25:38.599711 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:38.599641    2584 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"
Jul 24 14:25:38.665111 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:38.665085    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:38.665111 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:38.665101    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:38.665298 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:38.665099    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:38.665298 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:38.665199    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:38.665298 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:38.665257    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:38.665395 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:38.665314    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:39.100661 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:39.100633    2584 generic.go:356] "Generic (PLEG): container finished" podID="100842d2-32ef-4422-b1ef-039e00c95e3e" containerID="6c0717a499d92dc0e06e25a3dfb47709f0f7d524a3e4cb7b45759fa6780c6d18" exitCode=0
Jul 24 14:25:39.100804 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:39.100690    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerDied","Data":"6c0717a499d92dc0e06e25a3dfb47709f0f7d524a3e4cb7b45759fa6780c6d18"}
Jul 24 14:25:40.664919 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:40.664844    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:40.664919 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:40.664868    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:40.665316 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:40.664845    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:40.665316 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:40.664948    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:40.665316 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:40.665025    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:40.665316 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:40.665119    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:41.369075 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:41.369040    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:41.369245 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:41.369086    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:41.369245 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.369195    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:41.369245 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.369197    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:41.369394 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.369260    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs podName:7478e2db-f305-4c13-8f1b-2e881537d6a4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:13.369242473 +0000 UTC m=+230.153942916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs") pod "network-metrics-daemon-4gdl8" (UID: "7478e2db-f305-4c13-8f1b-2e881537d6a4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 14:25:41.369394 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.369279    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret podName:8d36989a-3602-4e05-8ab1-7f85407fd61c nodeName:}" failed. No retries permitted until 2026-07-24 14:26:13.369269874 +0000 UTC m=+230.153970321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret") pod "global-pull-secret-syncer-wz7zb" (UID: "8d36989a-3602-4e05-8ab1-7f85407fd61c") : object "kube-system"/"original-pull-secret" not registered
Jul 24 14:25:41.469917 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:41.469880    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:41.470082 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.470038    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 14:25:41.470082 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.470054    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 14:25:41.470082 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.470062    2584 projected.go:196] Error preparing data for projected volume kube-api-access-qbnm2 for pod openshift-network-diagnostics/network-check-target-jfv8z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:41.470223 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:41.470108    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2 podName:d9e1f31f-b9b2-418a-8d01-399553e9f5a8 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:13.470094782 +0000 UTC m=+230.254795224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qbnm2" (UniqueName: "kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2") pod "network-check-target-jfv8z" (UID: "d9e1f31f-b9b2-418a-8d01-399553e9f5a8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 14:25:42.665096 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:42.665063    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:42.665567 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:42.665063    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:42.665567 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:42.665192    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wz7zb" podUID="8d36989a-3602-4e05-8ab1-7f85407fd61c"
Jul 24 14:25:42.665567 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:42.665266    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jfv8z" podUID="d9e1f31f-b9b2-418a-8d01-399553e9f5a8"
Jul 24 14:25:42.665567 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:42.665063    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:42.665567 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:42.665368    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gdl8" podUID="7478e2db-f305-4c13-8f1b-2e881537d6a4"
Jul 24 14:25:44.665284 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.665021    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:25:44.665662 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.665044    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:25:44.665662 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.665215    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:25:44.665662 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:44.665464    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:25:44.668848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.668816    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 14:25:44.668963 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.668883    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 14:25:44.670206 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.670186    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 14:25:44.670206 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.670204    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rbcqt\""
Jul 24 14:25:44.670341 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.670195    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 14:25:44.670341 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.670204    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xh5fm\""
Jul 24 14:25:44.946068 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:44.946003    2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-93.ec2.internal" event="NodeReady"
Jul 24 14:25:45.005078 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.005051    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"]
Jul 24 14:25:45.017352 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.017332    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"]
Jul 24 14:25:45.017482 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.017463    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.026266 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.026249    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"]
Jul 24 14:25:45.026374 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.026359    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.031113 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.031085    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 24 14:25:45.031113 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.031104    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.031291 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.031154    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.035325 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.035299    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6754854769-k8g7q"]
Jul 24 14:25:45.035409 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.035398    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.041732 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.041566    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.041850 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.041745    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 24 14:25:45.045621 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.045605    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-df9867497-4w8nq"]
Jul 24 14:25:45.045760 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.045740    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.054520 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.054493    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 24 14:25:45.054618 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.054523    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.056052 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.056021    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.057052 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.056729    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-j9lx2"]
Jul 24 14:25:45.057905 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.057141    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.059634 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.058057    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-clqrz\""
Jul 24 14:25:45.061341 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.060581    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 14:25:45.061341 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.060744    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-rnb9x\""
Jul 24 14:25:45.061341 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.060751    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 14:25:45.062043 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.062024    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 14:25:45.062144 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.062117    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 14:25:45.062606 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.062227    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7tkvk\""
Jul 24 14:25:45.062746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.062614    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6jc6f\""
Jul 24 14:25:45.063384 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.063359    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 14:25:45.064045 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.064029    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-md9bf\""
Jul 24 14:25:45.064133 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.064118    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.067421 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.067402    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 14:25:45.069226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.069208    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"]
Jul 24 14:25:45.069324 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.069275    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"]
Jul 24 14:25:45.069387 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.069326    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.076100 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.076081    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"]
Jul 24 14:25:45.076232 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.076214    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.080354 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.080307    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.080588 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.080558    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 24 14:25:45.080748 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.080571    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 24 14:25:45.081148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.080654    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 24 14:25:45.081148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.080684    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 24 14:25:45.081148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.080701    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.081429 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081354    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 24 14:25:45.081697 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081435    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.081697 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081533    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wfg8h\""
Jul 24 14:25:45.081697 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081688    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.081879 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081581    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.081879 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081590    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hps7c\""
Jul 24 14:25:45.081879 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081608    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 24 14:25:45.082022 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081572    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 24 14:25:45.082022 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.081992    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 24 14:25:45.082125 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.082074    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.082567 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.082534    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"]
Jul 24 14:25:45.082730 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.082717    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.087528 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.087511    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mn6gh\""
Jul 24 14:25:45.088763 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.088663    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 24 14:25:45.088891 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.088766    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 24 14:25:45.093485 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.093462    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"]
Jul 24 14:25:45.093930 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.093908    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 24 14:25:45.094023 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.093979    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.099300 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.099279    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pppdj\""
Jul 24 14:25:45.099384 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.099289    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 14:25:45.099608 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.099592    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 14:25:45.099669 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.099298    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.099877 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.099858    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.108483 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.108467    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"]
Jul 24 14:25:45.108600 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.108587    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"
Jul 24 14:25:45.112118 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.112101    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nvqkq\""
Jul 24 14:25:45.116554 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.116535    2584 generic.go:356] "Generic (PLEG): container finished" podID="100842d2-32ef-4422-b1ef-039e00c95e3e" containerID="efccdcfd3cd4d182bfef97b1f99aeb262f65bea109db59eb50759a35307fb719" exitCode=0
Jul 24 14:25:45.123026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.123010    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"]
Jul 24 14:25:45.123138 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.123124    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.127124 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.127106    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 14:25:45.133826 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.133808    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerDied","Data":"efccdcfd3cd4d182bfef97b1f99aeb262f65bea109db59eb50759a35307fb719"}
Jul 24 14:25:45.133922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.133857    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"]
Jul 24 14:25:45.133922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.133916    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.136984 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.136965    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 14:25:45.137183 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.137168    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 14:25:45.137238 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.137182    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 14:25:45.137429 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.137415    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 14:25:45.145120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.145102    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fxg9p"]
Jul 24 14:25:45.145297 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.145279    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"
Jul 24 14:25:45.154178 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.154159    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.154407 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.154382    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bphv4\""
Jul 24 14:25:45.154601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.154578    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.156167 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156139    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.156167 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156150    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"]
Jul 24 14:25:45.156332 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156225    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-j9lx2"]
Jul 24 14:25:45.156332 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156257    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-df9867497-4w8nq"]
Jul 24 14:25:45.156433 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156356    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6754854769-k8g7q"]
Jul 24 14:25:45.156433 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156386    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"]
Jul 24 14:25:45.156706 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156407    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"]
Jul 24 14:25:45.156779 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.156721    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ssbjq"]
Jul 24 14:25:45.165072 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.165053    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 24 14:25:45.165396 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.165163    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 24 14:25:45.165396 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.165181    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.165646 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.165626    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.166226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.166209    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gk4cx\""
Jul 24 14:25:45.169381 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.169362    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"]
Jul 24 14:25:45.169470 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.169398    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"]
Jul 24 14:25:45.169470 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.169427    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbkpq"]
Jul 24 14:25:45.169575 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.169510    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.174667 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.174650    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 14:25:45.174929 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.174913    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5jncb\""
Jul 24 14:25:45.174998 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.174932    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 14:25:45.180383 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.180363    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 24 14:25:45.182951 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.182935    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"]
Jul 24 14:25:45.183025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.182963    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"]
Jul 24 14:25:45.183025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.182979    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"]
Jul 24 14:25:45.183025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.182992    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ssbjq"]
Jul 24 14:25:45.183025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.183007    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"]
Jul 24 14:25:45.183025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.183019    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbkpq"]
Jul 24 14:25:45.183192 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.183031    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fxg9p"]
Jul 24 14:25:45.183192 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.183040    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.192048 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.192029    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 14:25:45.192404 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.192389    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 14:25:45.192485 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.192468    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6vcpp\""
Jul 24 14:25:45.198651 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198634    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d4ee50f-92ce-43fe-92f2-69391d789624-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.198751 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198657    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.198751 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198675    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrk97\" (UniqueName: \"kubernetes.io/projected/ed673a39-32ca-4685-b89c-f5de447ea0c8-kube-api-access-vrk97\") pod \"managed-serviceaccount-addon-agent-776c947c66-5gpxn\" (UID: \"ed673a39-32ca-4685-b89c-f5de447ea0c8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.198751 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198693    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09f2c2a-eb19-440c-8cf2-24d6f4328003-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.198751 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198711    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl92v\" (UniqueName: \"kubernetes.io/projected/5d4ee50f-92ce-43fe-92f2-69391d789624-kube-api-access-gl92v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.198932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75d26\" (UniqueName: \"kubernetes.io/projected/07b242ff-baae-4556-9d53-c483938af05e-kube-api-access-75d26\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.198932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09f2c2a-eb19-440c-8cf2-24d6f4328003-service-ca-bundle\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.198932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198861    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.198932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198888    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.198932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198912    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-trusted-ca\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a09f2c2a-eb19-440c-8cf2-24d6f4328003-serving-cert\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.199120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.198992    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.199120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199031    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmqz7\" (UniqueName: \"kubernetes.io/projected/c46b45e0-5174-4a77-9949-e7fe45501f2b-kube-api-access-qmqz7\") pod \"network-check-source-7d88cb55d8-hmpdp\" (UID: \"c46b45e0-5174-4a77-9949-e7fe45501f2b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"
Jul 24 14:25:45.199120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199050    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-image-registry-private-configuration\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199066    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-default-certificate\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.199120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199081    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a09f2c2a-eb19-440c-8cf2-24d6f4328003-tmp\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199147    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed673a39-32ca-4685-b89c-f5de447ea0c8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-776c947c66-5gpxn\" (UID: \"ed673a39-32ca-4685-b89c-f5de447ea0c8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d4ee50f-92ce-43fe-92f2-69391d789624-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199208    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-installation-pull-secrets\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gwrk\" (UniqueName: \"kubernetes.io/projected/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-kube-api-access-4gwrk\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a09f2c2a-eb19-440c-8cf2-24d6f4328003-snapshots\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199298    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199321    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/07b242ff-baae-4556-9d53-c483938af05e-config\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199355    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxc5w\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-kube-api-access-gxc5w\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199400    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0522bad-1a97-410d-bbb2-0037cdc9852d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199428    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b242ff-baae-4556-9d53-c483938af05e-serving-cert\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.199454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199455    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rttps\" (UniqueName: \"kubernetes.io/projected/a09f2c2a-eb19-440c-8cf2-24d6f4328003-kube-api-access-rttps\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199487    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb8ck\" (UniqueName: \"kubernetes.io/projected/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-kube-api-access-wb8ck\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199517    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-certificates\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199541    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-stats-auth\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199557    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lrcx\" (UniqueName: \"kubernetes.io/projected/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-kube-api-access-2lrcx\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199576    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-bound-sa-token\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199596    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199617    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.199848 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.199633    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4f73506-ad36-43a8-8685-b5daf2987c1f-ca-trust-extracted\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.205746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.205730    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 14:25:45.300051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300026    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-ca\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.300155 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pgtj\" (UniqueName: \"kubernetes.io/projected/5b52fc62-1a70-435a-9ccc-11d17011c038-kube-api-access-8pgtj\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.300155 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmqz7\" (UniqueName: \"kubernetes.io/projected/c46b45e0-5174-4a77-9949-e7fe45501f2b-kube-api-access-qmqz7\") pod \"network-check-source-7d88cb55d8-hmpdp\" (UID: \"c46b45e0-5174-4a77-9949-e7fe45501f2b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"
Jul 24 14:25:45.300155 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-tmp-dir\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.300255 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-image-registry-private-configuration\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.300255 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300172    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-config-volume\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.300255 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300201    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-default-certificate\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.300255 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300224    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a09f2c2a-eb19-440c-8cf2-24d6f4328003-tmp\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.300255 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300245    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-hub\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.300438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300266    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b52fc62-1a70-435a-9ccc-11d17011c038-klusterlet-config\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.300438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300308    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jmz9\" (UniqueName: \"kubernetes.io/projected/9c2c4b78-ceef-45aa-aab3-bac4403e834d-kube-api-access-8jmz9\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.300438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300334    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed673a39-32ca-4685-b89c-f5de447ea0c8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-776c947c66-5gpxn\" (UID: \"ed673a39-32ca-4685-b89c-f5de447ea0c8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.300438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300359    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-serving-cert\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.300438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300408    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-trusted-ca\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.300602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300458    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d4ee50f-92ce-43fe-92f2-69391d789624-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.300602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-installation-pull-secrets\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.300602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300515    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2c4b78-ceef-45aa-aab3-bac4403e834d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.300602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300542    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzzdb\" (UniqueName: \"kubernetes.io/projected/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-kube-api-access-vzzdb\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.300602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300570    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gwrk\" (UniqueName: \"kubernetes.io/projected/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-kube-api-access-4gwrk\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.300602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300597    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a09f2c2a-eb19-440c-8cf2-24d6f4328003-snapshots\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.300843 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300628    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.300843 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300595    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a09f2c2a-eb19-440c-8cf2-24d6f4328003-tmp\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.300843 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/07b242ff-baae-4556-9d53-c483938af05e-config\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.300843 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.300707    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 14:25:45.300843 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.300718    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxc5w\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-kube-api-access-gxc5w\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.300843 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.300776    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.800756416 +0000 UTC m=+202.585456869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : secret "router-metrics-certs-default" not found
Jul 24 14:25:45.301375 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301275    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/07b242ff-baae-4556-9d53-c483938af05e-config\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.301712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h79sd\" (UniqueName: \"kubernetes.io/projected/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-kube-api-access-h79sd\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.301712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301700    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0522bad-1a97-410d-bbb2-0037cdc9852d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.301887 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301726    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b242ff-baae-4556-9d53-c483938af05e-serving-cert\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.301887 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.301887 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301778    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rttps\" (UniqueName: \"kubernetes.io/projected/a09f2c2a-eb19-440c-8cf2-24d6f4328003-kube-api-access-rttps\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.301887 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301798    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.301887 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301852    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb8ck\" (UniqueName: \"kubernetes.io/projected/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-kube-api-access-wb8ck\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-certificates\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-config\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301941    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301663    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a09f2c2a-eb19-440c-8cf2-24d6f4328003-snapshots\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.301977    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-stats-auth\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302013    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lrcx\" (UniqueName: \"kubernetes.io/projected/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-kube-api-access-2lrcx\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302043    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-bound-sa-token\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302071    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302096    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.302137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4f73506-ad36-43a8-8685-b5daf2987c1f-ca-trust-extracted\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302146    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d4ee50f-92ce-43fe-92f2-69391d789624-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302168    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrk97\" (UniqueName: \"kubernetes.io/projected/ed673a39-32ca-4685-b89c-f5de447ea0c8-kube-api-access-vrk97\") pod \"managed-serviceaccount-addon-agent-776c947c66-5gpxn\" (UID: \"ed673a39-32ca-4685-b89c-f5de447ea0c8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302217    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09f2c2a-eb19-440c-8cf2-24d6f4328003-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302257    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl92v\" (UniqueName: \"kubernetes.io/projected/5d4ee50f-92ce-43fe-92f2-69391d789624-kube-api-access-gl92v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302299    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75d26\" (UniqueName: \"kubernetes.io/projected/07b242ff-baae-4556-9d53-c483938af05e-kube-api-access-75d26\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302427    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09f2c2a-eb19-440c-8cf2-24d6f4328003-service-ca-bundle\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302457    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5sdn\" (UniqueName: \"kubernetes.io/projected/ea5c9656-f807-497c-8f41-715f74b4385f-kube-api-access-p5sdn\") pod \"ingress-canary-gbkpq\" (UID: \"ea5c9656-f807-497c-8f41-715f74b4385f\") " pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302489    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.302601 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.302600    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.802582692 +0000 UTC m=+202.587283134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : configmap references non-existent config key: service-ca.crt
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302755    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-trusted-ca\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302783    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b52fc62-1a70-435a-9ccc-11d17011c038-tmp\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.302808    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc9bf\" (UniqueName: \"kubernetes.io/projected/0668758d-4f81-4136-a78a-1f170490d401-kube-api-access-pc9bf\") pod \"volume-data-source-validator-549c6d54cf-hcrwc\" (UID: \"0668758d-4f81-4136-a78a-1f170490d401\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.302915    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.302968    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls podName:a64d93b6-1a8a-4094-9e1f-d2449c1bcf67 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.80295403 +0000 UTC m=+202.587654474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls") pod "cluster-samples-operator-6748489656-nrlqz" (UID: "a64d93b6-1a8a-4094-9e1f-d2449c1bcf67") : secret "samples-operator-tls" not found
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.303005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a09f2c2a-eb19-440c-8cf2-24d6f4328003-serving-cert\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.303032    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.303148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.303058    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea5c9656-f807-497c-8f41-715f74b4385f-cert\") pod \"ingress-canary-gbkpq\" (UID: \"ea5c9656-f807-497c-8f41-715f74b4385f\") " pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.304202 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.303647    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09f2c2a-eb19-440c-8cf2-24d6f4328003-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.304202 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.303900    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-trusted-ca\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.304202 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.303914    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4f73506-ad36-43a8-8685-b5daf2987c1f-ca-trust-extracted\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.304392 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.304250    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:25:45.304392 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.304265    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6754854769-k8g7q: secret "image-registry-tls" not found
Jul 24 14:25:45.304392 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.304317    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls podName:c4f73506-ad36-43a8-8685-b5daf2987c1f nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.804300813 +0000 UTC m=+202.589001256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls") pod "image-registry-6754854769-k8g7q" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f") : secret "image-registry-tls" not found
Jul 24 14:25:45.304559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.304418    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d4ee50f-92ce-43fe-92f2-69391d789624-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.304559 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.304504    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 14:25:45.304669 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.304566    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert podName:d0522bad-1a97-410d-bbb2-0037cdc9852d nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.804549197 +0000 UTC m=+202.589249651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-k6dbg" (UID: "d0522bad-1a97-410d-bbb2-0037cdc9852d") : secret "networking-console-plugin-cert" not found
Jul 24 14:25:45.304878 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.304856    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d4ee50f-92ce-43fe-92f2-69391d789624-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.305472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.305120    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-installation-pull-secrets\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.305472 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.305218    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:45.305472 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.305271    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls podName:6852cf8e-b13e-49e3-b9bb-383a5ea7e803 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.805256621 +0000 UTC m=+202.589957068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kschn" (UID: "6852cf8e-b13e-49e3-b9bb-383a5ea7e803") : secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:45.305472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.305316    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-default-certificate\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.305472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.305358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-certificates\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.305472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.305419    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-image-registry-private-configuration\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.305861 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.305764    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09f2c2a-eb19-440c-8cf2-24d6f4328003-service-ca-bundle\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.305913 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.305854    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.306155 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.306137    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-stats-auth\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.306467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.306446    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a09f2c2a-eb19-440c-8cf2-24d6f4328003-serving-cert\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.306959 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.306938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/07b242ff-baae-4556-9d53-c483938af05e-serving-cert\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.307971 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.307957    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0522bad-1a97-410d-bbb2-0037cdc9852d-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.310514 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.310498    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ed673a39-32ca-4685-b89c-f5de447ea0c8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-776c947c66-5gpxn\" (UID: \"ed673a39-32ca-4685-b89c-f5de447ea0c8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.310999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.310983    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmqz7\" (UniqueName: \"kubernetes.io/projected/c46b45e0-5174-4a77-9949-e7fe45501f2b-kube-api-access-qmqz7\") pod \"network-check-source-7d88cb55d8-hmpdp\" (UID: \"c46b45e0-5174-4a77-9949-e7fe45501f2b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"
Jul 24 14:25:45.313150 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.313124    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxc5w\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-kube-api-access-gxc5w\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.313932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.313911    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrk97\" (UniqueName: \"kubernetes.io/projected/ed673a39-32ca-4685-b89c-f5de447ea0c8-kube-api-access-vrk97\") pod \"managed-serviceaccount-addon-agent-776c947c66-5gpxn\" (UID: \"ed673a39-32ca-4685-b89c-f5de447ea0c8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.319014 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.318994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gwrk\" (UniqueName: \"kubernetes.io/projected/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-kube-api-access-4gwrk\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.323363 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.323325    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-bound-sa-token\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.323748 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.323729    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75d26\" (UniqueName: \"kubernetes.io/projected/07b242ff-baae-4556-9d53-c483938af05e-kube-api-access-75d26\") pod \"service-ca-operator-5d6c75f84f-st2k9\" (UID: \"07b242ff-baae-4556-9d53-c483938af05e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.325712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.325672    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl92v\" (UniqueName: \"kubernetes.io/projected/5d4ee50f-92ce-43fe-92f2-69391d789624-kube-api-access-gl92v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c\" (UID: \"5d4ee50f-92ce-43fe-92f2-69391d789624\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.325991 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.325975    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb8ck\" (UniqueName: \"kubernetes.io/projected/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-kube-api-access-wb8ck\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.326070 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.326038    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rttps\" (UniqueName: \"kubernetes.io/projected/a09f2c2a-eb19-440c-8cf2-24d6f4328003-kube-api-access-rttps\") pod \"insights-operator-5885fdb759-j9lx2\" (UID: \"a09f2c2a-eb19-440c-8cf2-24d6f4328003\") " pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.327352 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.327334    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lrcx\" (UniqueName: \"kubernetes.io/projected/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-kube-api-access-2lrcx\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.355860 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.355823    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:45.380118 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.380098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-j9lx2"
Jul 24 14:25:45.388775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.388760    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"
Jul 24 14:25:45.404286 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404266    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-config\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.404376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.404429 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.404416    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:25:45.404489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404470    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5sdn\" (UniqueName: \"kubernetes.io/projected/ea5c9656-f807-497c-8f41-715f74b4385f-kube-api-access-p5sdn\") pod \"ingress-canary-gbkpq\" (UID: \"ea5c9656-f807-497c-8f41-715f74b4385f\") " pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.404489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.404476    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls podName:c872d451-23b6-49e7-b8b0-a41cdcbb4ff5 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:45.904459028 +0000 UTC m=+202.689159473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls") pod "dns-default-ssbjq" (UID: "c872d451-23b6-49e7-b8b0-a41cdcbb4ff5") : secret "dns-default-metrics-tls" not found
Jul 24 14:25:45.404594 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404541    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b52fc62-1a70-435a-9ccc-11d17011c038-tmp\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.404594 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc9bf\" (UniqueName: \"kubernetes.io/projected/0668758d-4f81-4136-a78a-1f170490d401-kube-api-access-pc9bf\") pod \"volume-data-source-validator-549c6d54cf-hcrwc\" (UID: \"0668758d-4f81-4136-a78a-1f170490d401\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"
Jul 24 14:25:45.404690 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404607    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea5c9656-f807-497c-8f41-715f74b4385f-cert\") pod \"ingress-canary-gbkpq\" (UID: \"ea5c9656-f807-497c-8f41-715f74b4385f\") " pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.404690 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404639    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-ca\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.404690 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404673    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pgtj\" (UniqueName: \"kubernetes.io/projected/5b52fc62-1a70-435a-9ccc-11d17011c038-kube-api-access-8pgtj\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.404849 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404699    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-tmp-dir\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.405022 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.404996    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-config-volume\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.405146 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405039    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-config\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.405146 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-hub\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.405146 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405094    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b52fc62-1a70-435a-9ccc-11d17011c038-klusterlet-config\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.405146 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405141    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jmz9\" (UniqueName: \"kubernetes.io/projected/9c2c4b78-ceef-45aa-aab3-bac4403e834d-kube-api-access-8jmz9\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.405339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405150    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b52fc62-1a70-435a-9ccc-11d17011c038-tmp\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.405339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405172    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-serving-cert\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.405339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405197    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-trusted-ca\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.405339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405236    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2c4b78-ceef-45aa-aab3-bac4403e834d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.405339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405263    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzzdb\" (UniqueName: \"kubernetes.io/projected/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-kube-api-access-vzzdb\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.405607 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405339    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h79sd\" (UniqueName: \"kubernetes.io/projected/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-kube-api-access-h79sd\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.405607 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405373    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.405607 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405568    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-config-volume\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.406003 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.405980    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c2c4b78-ceef-45aa-aab3-bac4403e834d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.406087 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.406032    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.407057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.406675    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-trusted-ca\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.407057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.406994    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-tmp-dir\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.408094 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.408069    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-ca\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.408374 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.408352    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5b52fc62-1a70-435a-9ccc-11d17011c038-klusterlet-config\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.408662 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.408634    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-serving-cert\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.408760 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.408736    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.408811 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.408754    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.409850 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.409803    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea5c9656-f807-497c-8f41-715f74b4385f-cert\") pod \"ingress-canary-gbkpq\" (UID: \"ea5c9656-f807-497c-8f41-715f74b4385f\") " pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.410406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.410386    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9c2c4b78-ceef-45aa-aab3-bac4403e834d-hub\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.415083 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.415067    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:25:45.420625 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.420601    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc9bf\" (UniqueName: \"kubernetes.io/projected/0668758d-4f81-4136-a78a-1f170490d401-kube-api-access-pc9bf\") pod \"volume-data-source-validator-549c6d54cf-hcrwc\" (UID: \"0668758d-4f81-4136-a78a-1f170490d401\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"
Jul 24 14:25:45.422224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.422198    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5sdn\" (UniqueName: \"kubernetes.io/projected/ea5c9656-f807-497c-8f41-715f74b4385f-kube-api-access-p5sdn\") pod \"ingress-canary-gbkpq\" (UID: \"ea5c9656-f807-497c-8f41-715f74b4385f\") " pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.422350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.422262    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"
Jul 24 14:25:45.427149 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.427130    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzzdb\" (UniqueName: \"kubernetes.io/projected/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-kube-api-access-vzzdb\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.427981 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.427962    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h79sd\" (UniqueName: \"kubernetes.io/projected/74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7-kube-api-access-h79sd\") pod \"console-operator-584dcb6b54-fxg9p\" (UID: \"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7\") " pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.428299 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.428279    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jmz9\" (UniqueName: \"kubernetes.io/projected/9c2c4b78-ceef-45aa-aab3-bac4403e834d-kube-api-access-8jmz9\") pod \"cluster-proxy-proxy-agent-b594c5f58-w6788\" (UID: \"9c2c4b78-ceef-45aa-aab3-bac4403e834d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.429877 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.429819    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pgtj\" (UniqueName: \"kubernetes.io/projected/5b52fc62-1a70-435a-9ccc-11d17011c038-kube-api-access-8pgtj\") pod \"klusterlet-addon-workmgr-788fb66f64-bb9wm\" (UID: \"5b52fc62-1a70-435a-9ccc-11d17011c038\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.430499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.430463    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:45.447904 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.447516    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:25:45.470107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.470023    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"
Jul 24 14:25:45.477788 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.477698    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:25:45.509107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.506989    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbkpq"
Jul 24 14:25:45.543386 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.542663    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:45.623237 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.623088    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-j9lx2"]
Jul 24 14:25:45.626028 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.625766    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"
Jul 24 14:25:45.629077 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.627723    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09f2c2a_eb19_440c_8cf2_24d6f4328003.slice/crio-c47f093ee90b7e57e7ee282373b9953c8bb03345efaf3f663bceacc7741c1ca6 WatchSource:0}: Error finding container c47f093ee90b7e57e7ee282373b9953c8bb03345efaf3f663bceacc7741c1ca6: Status 404 returned error can't find the container with id c47f093ee90b7e57e7ee282373b9953c8bb03345efaf3f663bceacc7741c1ca6
Jul 24 14:25:45.688522 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.685605    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9"]
Jul 24 14:25:45.693689 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.693632    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"]
Jul 24 14:25:45.700942 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.700264    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"]
Jul 24 14:25:45.701252 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.701221    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded673a39_32ca_4685_b89c_f5de447ea0c8.slice/crio-2c6ec5f19c5a27fa234ee628dfd64a2a16c7378c10940d946180a35277037389 WatchSource:0}: Error finding container 2c6ec5f19c5a27fa234ee628dfd64a2a16c7378c10940d946180a35277037389: Status 404 returned error can't find the container with id 2c6ec5f19c5a27fa234ee628dfd64a2a16c7378c10940d946180a35277037389
Jul 24 14:25:45.702650 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.702608    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b52fc62_1a70_435a_9ccc_11d17011c038.slice/crio-f94f65e5174cb967f3dc169af3c1e1e70f30324967639d7e46ce2a1ff1c84508 WatchSource:0}: Error finding container f94f65e5174cb967f3dc169af3c1e1e70f30324967639d7e46ce2a1ff1c84508: Status 404 returned error can't find the container with id f94f65e5174cb967f3dc169af3c1e1e70f30324967639d7e46ce2a1ff1c84508
Jul 24 14:25:45.712500 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.712435    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp"]
Jul 24 14:25:45.715178 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.715134    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:45.740290 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.740267    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"]
Jul 24 14:25:45.742099 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.742075    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c2c4b78_ceef_45aa_aab3_bac4403e834d.slice/crio-f744eaa7b2c02812a53df89e7a108856c5e68e5a02791989eae7e99bb8cc3bd6 WatchSource:0}: Error finding container f744eaa7b2c02812a53df89e7a108856c5e68e5a02791989eae7e99bb8cc3bd6: Status 404 returned error can't find the container with id f744eaa7b2c02812a53df89e7a108856c5e68e5a02791989eae7e99bb8cc3bd6
Jul 24 14:25:45.752697 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.752675    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc"]
Jul 24 14:25:45.755996 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.755931    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0668758d_4f81_4136_a78a_1f170490d401.slice/crio-ebc7af0d3b887fd8d9d1e290af75f6061f63fd18d04b5a5deae24674609183db WatchSource:0}: Error finding container ebc7af0d3b887fd8d9d1e290af75f6061f63fd18d04b5a5deae24674609183db: Status 404 returned error can't find the container with id ebc7af0d3b887fd8d9d1e290af75f6061f63fd18d04b5a5deae24674609183db
Jul 24 14:25:45.765887 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.765863    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fxg9p"]
Jul 24 14:25:45.768626 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.768604    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e9f2bd_c80c_49d5_a2ee_19e94fc8ddd7.slice/crio-b8479a2b078e20203d783e3e48929bb6c50968ce41eda1551170da4d00a48f19 WatchSource:0}: Error finding container b8479a2b078e20203d783e3e48929bb6c50968ce41eda1551170da4d00a48f19: Status 404 returned error can't find the container with id b8479a2b078e20203d783e3e48929bb6c50968ce41eda1551170da4d00a48f19
Jul 24 14:25:45.773449 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.773428    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbkpq"]
Jul 24 14:25:45.774782 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.774762    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5c9656_f807_497c_8f41_715f74b4385f.slice/crio-a13acc38f080eadee283eef94a1f92da8394f953fcb17bee93c7b874f16832ea WatchSource:0}: Error finding container a13acc38f080eadee283eef94a1f92da8394f953fcb17bee93c7b874f16832ea: Status 404 returned error can't find the container with id a13acc38f080eadee283eef94a1f92da8394f953fcb17bee93c7b874f16832ea
Jul 24 14:25:45.813093 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.813061    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c"]
Jul 24 14:25:45.816421 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.816402    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.816480 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.816464    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:45.816512 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.816485    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:45.816572 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816553    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 14:25:45.816618 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.816571    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:45.816618 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.816590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:45.816618 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816599    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:25:45.816618 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816612    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6754854769-k8g7q: secret "image-registry-tls" not found
Jul 24 14:25:45.816618 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816614    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.816595765 +0000 UTC m=+203.601296210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : secret "router-metrics-certs-default" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816629    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816555    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816651    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls podName:c4f73506-ad36-43a8-8685-b5daf2987c1f nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.816639363 +0000 UTC m=+203.601339809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls") pod "image-registry-6754854769-k8g7q" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f") : secret "image-registry-tls" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816670    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls podName:a64d93b6-1a8a-4094-9e1f-d2449c1bcf67 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.816658296 +0000 UTC m=+203.601358739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls") pod "cluster-samples-operator-6748489656-nrlqz" (UID: "a64d93b6-1a8a-4094-9e1f-d2449c1bcf67") : secret "samples-operator-tls" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.816692    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816735    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.816723118 +0000 UTC m=+203.601423563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : configmap references non-existent config key: service-ca.crt
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816735    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816756    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls podName:6852cf8e-b13e-49e3-b9bb-383a5ea7e803 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.816746697 +0000 UTC m=+203.601447139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kschn" (UID: "6852cf8e-b13e-49e3-b9bb-383a5ea7e803") : secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:45.816805 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.816775    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert podName:d0522bad-1a97-410d-bbb2-0037cdc9852d nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.816767391 +0000 UTC m=+203.601467834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-k6dbg" (UID: "d0522bad-1a97-410d-bbb2-0037cdc9852d") : secret "networking-console-plugin-cert" not found
Jul 24 14:25:45.825293 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:25:45.825270    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d4ee50f_92ce_43fe_92f2_69391d789624.slice/crio-a51b0b7cff0b06714af17b36d462c9423e83a86c47f1cbc539c80dff00094eef WatchSource:0}: Error finding container a51b0b7cff0b06714af17b36d462c9423e83a86c47f1cbc539c80dff00094eef: Status 404 returned error can't find the container with id a51b0b7cff0b06714af17b36d462c9423e83a86c47f1cbc539c80dff00094eef
Jul 24 14:25:45.917258 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.917223    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:45.917374 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.917360    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:25:45.917422 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:45.917412    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls podName:c872d451-23b6-49e7-b8b0-a41cdcbb4ff5 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:46.917398374 +0000 UTC m=+203.702098817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls") pod "dns-default-ssbjq" (UID: "c872d451-23b6-49e7-b8b0-a41cdcbb4ff5") : secret "dns-default-metrics-tls" not found
Jul 24 14:25:45.937668 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:45.937640    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:46.119998 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.119966    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc" event={"ID":"0668758d-4f81-4136-a78a-1f170490d401","Type":"ContainerStarted","Data":"ebc7af0d3b887fd8d9d1e290af75f6061f63fd18d04b5a5deae24674609183db"}
Jul 24 14:25:46.121145 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.121125    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" event={"ID":"9c2c4b78-ceef-45aa-aab3-bac4403e834d","Type":"ContainerStarted","Data":"f744eaa7b2c02812a53df89e7a108856c5e68e5a02791989eae7e99bb8cc3bd6"}
Jul 24 14:25:46.122158 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.122133    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp" event={"ID":"c46b45e0-5174-4a77-9949-e7fe45501f2b","Type":"ContainerStarted","Data":"e5a885f27e87b82fa50db76f7c79d1cbe43161225e0c7c55389406db8510d910"}
Jul 24 14:25:46.123214 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.123195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" event={"ID":"ed673a39-32ca-4685-b89c-f5de447ea0c8","Type":"ContainerStarted","Data":"2c6ec5f19c5a27fa234ee628dfd64a2a16c7378c10940d946180a35277037389"}
Jul 24 14:25:46.124322 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.124303    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9" event={"ID":"07b242ff-baae-4556-9d53-c483938af05e","Type":"ContainerStarted","Data":"86b32177c64433b5efccbe94615f6703f24365360cd47924d32798248b3705b5"}
Jul 24 14:25:46.127128 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.127109    2584 generic.go:356] "Generic (PLEG): container finished" podID="100842d2-32ef-4422-b1ef-039e00c95e3e" containerID="339a6a004aa3fa594ff08ae12d8165270ac1575167d76aa70a0584d4b9602505" exitCode=0
Jul 24 14:25:46.127308 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.127189    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerDied","Data":"339a6a004aa3fa594ff08ae12d8165270ac1575167d76aa70a0584d4b9602505"}
Jul 24 14:25:46.128701 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.128670    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c" event={"ID":"5d4ee50f-92ce-43fe-92f2-69391d789624","Type":"ContainerStarted","Data":"a51b0b7cff0b06714af17b36d462c9423e83a86c47f1cbc539c80dff00094eef"}
Jul 24 14:25:46.132026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.132004    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbkpq" event={"ID":"ea5c9656-f807-497c-8f41-715f74b4385f","Type":"ContainerStarted","Data":"a13acc38f080eadee283eef94a1f92da8394f953fcb17bee93c7b874f16832ea"}
Jul 24 14:25:46.133001 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.132981    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" event={"ID":"5b52fc62-1a70-435a-9ccc-11d17011c038","Type":"ContainerStarted","Data":"f94f65e5174cb967f3dc169af3c1e1e70f30324967639d7e46ce2a1ff1c84508"}
Jul 24 14:25:46.133983 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.133954    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-j9lx2" event={"ID":"a09f2c2a-eb19-440c-8cf2-24d6f4328003","Type":"ContainerStarted","Data":"c47f093ee90b7e57e7ee282373b9953c8bb03345efaf3f663bceacc7741c1ca6"}
Jul 24 14:25:46.135015 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.134995    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" event={"ID":"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7","Type":"ContainerStarted","Data":"b8479a2b078e20203d783e3e48929bb6c50968ce41eda1551170da4d00a48f19"}
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.826659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.826769    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.826800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.826853    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.826879    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.826915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.827173    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.827234    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.827215293 +0000 UTC m=+205.611915735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : secret "router-metrics-certs-default" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.827921    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.827939    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6754854769-k8g7q: secret "image-registry-tls" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.827984    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls podName:c4f73506-ad36-43a8-8685-b5daf2987c1f nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.827969745 +0000 UTC m=+205.612670189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls") pod "image-registry-6754854769-k8g7q" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f") : secret "image-registry-tls" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828051    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828086    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls podName:6852cf8e-b13e-49e3-b9bb-383a5ea7e803 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.82807478 +0000 UTC m=+205.612775247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kschn" (UID: "6852cf8e-b13e-49e3-b9bb-383a5ea7e803") : secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828206    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.828139237 +0000 UTC m=+205.612839682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : configmap references non-existent config key: service-ca.crt
Jul 24 14:25:46.828489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828305    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 14:25:46.830307 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828339    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls podName:a64d93b6-1a8a-4094-9e1f-d2449c1bcf67 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.82832737 +0000 UTC m=+205.613027819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls") pod "cluster-samples-operator-6748489656-nrlqz" (UID: "a64d93b6-1a8a-4094-9e1f-d2449c1bcf67") : secret "samples-operator-tls" not found
Jul 24 14:25:46.830307 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828392    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 14:25:46.830307 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.828450    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert podName:d0522bad-1a97-410d-bbb2-0037cdc9852d nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.828438878 +0000 UTC m=+205.613139326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-k6dbg" (UID: "d0522bad-1a97-410d-bbb2-0037cdc9852d") : secret "networking-console-plugin-cert" not found
Jul 24 14:25:46.929246 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:46.928154    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:46.929246 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.928458    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:25:46.929246 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:46.928518    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls podName:c872d451-23b6-49e7-b8b0-a41cdcbb4ff5 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:48.928499619 +0000 UTC m=+205.713200063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls") pod "dns-default-ssbjq" (UID: "c872d451-23b6-49e7-b8b0-a41cdcbb4ff5") : secret "dns-default-metrics-tls" not found
Jul 24 14:25:47.080849 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:47.080756    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:47.168242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:47.168207    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54lzz" event={"ID":"100842d2-32ef-4422-b1ef-039e00c95e3e","Type":"ContainerStarted","Data":"650b0970220d1f4bc9669ab147b13b8a7acb1397828490db31b307c8c35ec8ad"}
Jul 24 14:25:47.201862 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:47.201814    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:47.228330 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:47.227633    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-54lzz" podStartSLOduration=12.415661551 podStartE2EDuration="43.227610892s" totalImagesPullingTime="30.811949341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:47.225804197 +0000 UTC m=+204.010504667" watchObservedRunningTime="2026-07-24 14:25:47.227610892 +0000 UTC m=+204.012311361"
Jul 24 14:25:47.441019 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:47.440940    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:47.545329 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:47.545295    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:48.849142 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.849105    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.849223    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.849256    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.849298    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.849324    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.849362    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.849516    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 14:25:48.849569 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.849574    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert podName:d0522bad-1a97-410d-bbb2-0037cdc9852d nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.849554549 +0000 UTC m=+209.634254995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-k6dbg" (UID: "d0522bad-1a97-410d-bbb2-0037cdc9852d") : secret "networking-console-plugin-cert" not found
Jul 24 14:25:48.849971 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.849953    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 14:25:48.850029 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850009    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.849993103 +0000 UTC m=+209.634693548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : secret "router-metrics-certs-default" not found
Jul 24 14:25:48.850086 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850075    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:25:48.850139 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850086    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6754854769-k8g7q: secret "image-registry-tls" not found
Jul 24 14:25:48.850139 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850118    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls podName:c4f73506-ad36-43a8-8685-b5daf2987c1f nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.85010849 +0000 UTC m=+209.634808949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls") pod "image-registry-6754854769-k8g7q" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f") : secret "image-registry-tls" not found
Jul 24 14:25:48.850300 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850175    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:48.850300 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850207    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls podName:6852cf8e-b13e-49e3-b9bb-383a5ea7e803 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.85019637 +0000 UTC m=+209.634896813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kschn" (UID: "6852cf8e-b13e-49e3-b9bb-383a5ea7e803") : secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:48.850300 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850266    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.850256995 +0000 UTC m=+209.634957439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : configmap references non-existent config key: service-ca.crt
Jul 24 14:25:48.850465 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850322    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 14:25:48.850465 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.850355    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls podName:a64d93b6-1a8a-4094-9e1f-d2449c1bcf67 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.85034409 +0000 UTC m=+209.635044532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls") pod "cluster-samples-operator-6748489656-nrlqz" (UID: "a64d93b6-1a8a-4094-9e1f-d2449c1bcf67") : secret "samples-operator-tls" not found
Jul 24 14:25:48.951133 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:48.950445    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:48.951133 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.950666    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:25:48.951133 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:48.950729    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls podName:c872d451-23b6-49e7-b8b0-a41cdcbb4ff5 nodeName:}" failed. No retries permitted until 2026-07-24 14:25:52.95071006 +0000 UTC m=+209.735410520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls") pod "dns-default-ssbjq" (UID: "c872d451-23b6-49e7-b8b0-a41cdcbb4ff5") : secret "dns-default-metrics-tls" not found
Jul 24 14:25:49.194492 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:49.193728    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc" event={"ID":"0668758d-4f81-4136-a78a-1f170490d401","Type":"ContainerStarted","Data":"c2c52bf01b1469a7434324c5eaca2f7a54474d52e2da670446d50c0403f34299"}
Jul 24 14:25:49.220338 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:49.219332    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-hcrwc" podStartSLOduration=44.342638589 podStartE2EDuration="47.219311135s" totalImagesPullingTime="2.876672546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:49.219075127 +0000 UTC m=+206.003775592" watchObservedRunningTime="2026-07-24 14:25:49.219311135 +0000 UTC m=+206.004011600"
Jul 24 14:25:49.351216 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:49.351176    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:49.479549 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:49.479385    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:49.723865 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:49.723674    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:49.845993 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:49.845813    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:50.786561 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:50.786522    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:50.945226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:50.945163    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:51.667301 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:51.667262    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:51.783821 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:51.783784    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:51.885897 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:51.885862    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log" limit=1024
Jul 24 14:25:52.888542 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.888510    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.888630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.888659    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.888699    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888706    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.888723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.888756    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888774    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888790    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.88876841 +0000 UTC m=+217.673468871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : secret "router-metrics-certs-default" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888851    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888905    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert podName:d0522bad-1a97-410d-bbb2-0037cdc9852d nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.888888012 +0000 UTC m=+217.673588474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-k6dbg" (UID: "d0522bad-1a97-410d-bbb2-0037cdc9852d") : secret "networking-console-plugin-cert" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888921    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.888912371 +0000 UTC m=+217.673612814 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : configmap references non-existent config key: service-ca.crt
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888943    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888976    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888981    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls podName:6852cf8e-b13e-49e3-b9bb-383a5ea7e803 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.888968881 +0000 UTC m=+217.673669330 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kschn" (UID: "6852cf8e-b13e-49e3-b9bb-383a5ea7e803") : secret "cluster-monitoring-operator-tls" not found
Jul 24 14:25:52.889024 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.888792    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6754854769-k8g7q: secret "image-registry-tls" not found
Jul 24 14:25:52.889650 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.889005    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls podName:a64d93b6-1a8a-4094-9e1f-d2449c1bcf67 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.888996018 +0000 UTC m=+217.673696499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls") pod "cluster-samples-operator-6748489656-nrlqz" (UID: "a64d93b6-1a8a-4094-9e1f-d2449c1bcf67") : secret "samples-operator-tls" not found
Jul 24 14:25:52.889650 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.889020    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls podName:c4f73506-ad36-43a8-8685-b5daf2987c1f nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.889010714 +0000 UTC m=+217.673711162 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls") pod "image-registry-6754854769-k8g7q" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f") : secret "image-registry-tls" not found
Jul 24 14:25:52.989885 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:52.989856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:25:52.990034 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.989985    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:25:52.990078 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:52.990064    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls podName:c872d451-23b6-49e7-b8b0-a41cdcbb4ff5 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.990042199 +0000 UTC m=+217.774742644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls") pod "dns-default-ssbjq" (UID: "c872d451-23b6-49e7-b8b0-a41cdcbb4ff5") : secret "dns-default-metrics-tls" not found
Jul 24 14:25:59.224290 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.224251    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c" event={"ID":"5d4ee50f-92ce-43fe-92f2-69391d789624","Type":"ContainerStarted","Data":"9d8b13fb29bb7cb5cfde3c509d3434c60b09067a38c8cd0766a54f7a72176fab"}
Jul 24 14:25:59.225741 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.225715    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbkpq" event={"ID":"ea5c9656-f807-497c-8f41-715f74b4385f","Type":"ContainerStarted","Data":"ef4261a69af2c9540685dfbd896f21ad54b27c02fca2174b0367f0ea87792886"}
Jul 24 14:25:59.227602 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.227576    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" event={"ID":"5b52fc62-1a70-435a-9ccc-11d17011c038","Type":"ContainerStarted","Data":"349ef96a8eb3c6d803c4854071cd868a4dca38473fcc5ee290e545df0604d8e5"}
Jul 24 14:25:59.228298 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.228242    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:59.230015 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.229999    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:25:59.230376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.230327    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-j9lx2" event={"ID":"a09f2c2a-eb19-440c-8cf2-24d6f4328003","Type":"ContainerStarted","Data":"25248090ac620724030b90462e9cfe083be3f6a324eb31f444470b993a632cdf"}
Jul 24 14:25:59.232179 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.232156    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/0.log"
Jul 24 14:25:59.232264 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.232194    2584 generic.go:356] "Generic (PLEG): container finished" podID="74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7" containerID="710079a6ed9715309b8d786bc507492f904db41000cac1430fa8e6939cc8ec9d" exitCode=255
Jul 24 14:25:59.232328 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.232296    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" event={"ID":"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7","Type":"ContainerDied","Data":"710079a6ed9715309b8d786bc507492f904db41000cac1430fa8e6939cc8ec9d"}
Jul 24 14:25:59.232628 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.232613    2584 scope.go:117] "RemoveContainer" containerID="710079a6ed9715309b8d786bc507492f904db41000cac1430fa8e6939cc8ec9d"
Jul 24 14:25:59.234692 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.234672    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" event={"ID":"9c2c4b78-ceef-45aa-aab3-bac4403e834d","Type":"ContainerStarted","Data":"a0af074f163b62e6fec7aeaced56461ba1b5b41c1ddf9496db3ef8f56797c7b4"}
Jul 24 14:25:59.234800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.234705    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" event={"ID":"9c2c4b78-ceef-45aa-aab3-bac4403e834d","Type":"ContainerStarted","Data":"b583dc9f5e0f2fd2c006f240fb5d1675a784b24f251d164614195ca006316841"}
Jul 24 14:25:59.234800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.234722    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" event={"ID":"9c2c4b78-ceef-45aa-aab3-bac4403e834d","Type":"ContainerStarted","Data":"6b55f7f17a29c2ab3e97165d552fa2bb6ffcfc904be5733823adb09f4711f991"}
Jul 24 14:25:59.237705 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.237687    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp" event={"ID":"c46b45e0-5174-4a77-9949-e7fe45501f2b","Type":"ContainerStarted","Data":"29cfe5e04f94298c5cd21c8e37a91c9566c3917110ca6ad790620773a7ddf146"}
Jul 24 14:25:59.239251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.239233    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" event={"ID":"ed673a39-32ca-4685-b89c-f5de447ea0c8","Type":"ContainerStarted","Data":"e53d7ea770f4ffa15588b2434a8e9afbd0db508ed50e185ddd73195184a07ddd"}
Jul 24 14:25:59.240739 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.240719    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9" event={"ID":"07b242ff-baae-4556-9d53-c483938af05e","Type":"ContainerStarted","Data":"0e21f85d48f4c8ac3cd0776227ad63038dfa9eeb4ebbcf6127b7a5e5fd9e19d7"}
Jul 24 14:25:59.246616 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.246572    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c" podStartSLOduration=45.761725099 podStartE2EDuration="58.246554684s" totalImagesPullingTime="12.484829585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.245983961 +0000 UTC m=+216.030684423" watchObservedRunningTime="2026-07-24 14:25:59.246554684 +0000 UTC m=+216.031255127"
Jul 24 14:25:59.275163 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.275117    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" podStartSLOduration=44.685369588 podStartE2EDuration="57.275097401s" totalImagesPullingTime="12.589727813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.274043563 +0000 UTC m=+216.058744030" watchObservedRunningTime="2026-07-24 14:25:59.275097401 +0000 UTC m=+216.059797867"
Jul 24 14:25:59.405192 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.405137    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9" podStartSLOduration=44.763006647 podStartE2EDuration="57.405117374s" totalImagesPullingTime="12.642110727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.40322868 +0000 UTC m=+216.187929146" watchObservedRunningTime="2026-07-24 14:25:59.405117374 +0000 UTC m=+216.189817840"
Jul 24 14:25:59.437754 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.437372    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbkpq" podStartSLOduration=1.986574815 podStartE2EDuration="14.437350859s" totalImagesPullingTime="12.450776044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.436570635 +0000 UTC m=+216.221271102" watchObservedRunningTime="2026-07-24 14:25:59.437350859 +0000 UTC m=+216.222051325"
Jul 24 14:25:59.461540 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.461489    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" podStartSLOduration=45.829694074 podStartE2EDuration="58.461469126s" totalImagesPullingTime="12.631775052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.460932736 +0000 UTC m=+216.245633203" watchObservedRunningTime="2026-07-24 14:25:59.461469126 +0000 UTC m=+216.246169571"
Jul 24 14:25:59.488829 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.488740    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hmpdp" podStartSLOduration=44.865616851 podStartE2EDuration="57.488666915s" totalImagesPullingTime="12.623050064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.487369121 +0000 UTC m=+216.272069600" watchObservedRunningTime="2026-07-24 14:25:59.488666915 +0000 UTC m=+216.273367381"
Jul 24 14:25:59.515072 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.515019    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-j9lx2" podStartSLOduration=45.916759045 podStartE2EDuration="58.515001092s" totalImagesPullingTime="12.598242047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.513828342 +0000 UTC m=+216.298528805" watchObservedRunningTime="2026-07-24 14:25:59.515001092 +0000 UTC m=+216.299701558"
Jul 24 14:25:59.545354 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.545307    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" podStartSLOduration=45.913936042 podStartE2EDuration="58.545288827s" totalImagesPullingTime="12.631352785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:59.543283883 +0000 UTC m=+216.327984350" watchObservedRunningTime="2026-07-24 14:25:59.545288827 +0000 UTC m=+216.329989292"
Jul 24 14:25:59.665100 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:59.665054    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podUID="74ca4cffd660659a68e7ccdd89a85ecc"
Jul 24 14:25:59.755896 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.755787    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t6jpv"]
Jul 24 14:25:59.759317 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.759291    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.771626 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.771602    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 14:25:59.771767 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.771651    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 14:25:59.771861 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.771784    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w26gz\""
Jul 24 14:25:59.790382 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.790355    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t6jpv"]
Jul 24 14:25:59.857243 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.857212    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.857410 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.857251    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39a5116a-222d-4c6e-98b6-9ebc465096f4-crio-socket\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.857410 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.857283    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2plss\" (UniqueName: \"kubernetes.io/projected/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-api-access-2plss\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.857410 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.857380    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39a5116a-222d-4c6e-98b6-9ebc465096f4-data-volume\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.857567 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.857411    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958001 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.957972    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.958015    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39a5116a-222d-4c6e-98b6-9ebc465096f4-crio-socket\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.958034    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plss\" (UniqueName: \"kubernetes.io/projected/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-api-access-2plss\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.958078    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39a5116a-222d-4c6e-98b6-9ebc465096f4-data-volume\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.958103    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:59.958114    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.958129    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/39a5116a-222d-4c6e-98b6-9ebc465096f4-crio-socket\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.958172 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:59.958174    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.458155401 +0000 UTC m=+217.242855843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : configmap "kube-rbac-proxy" not found
Jul 24 14:25:59.958532 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:59.958312    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:25:59.958532 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:25:59.958343    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:00.458335165 +0000 UTC m=+217.243035607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:25:59.958532 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.958485    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/39a5116a-222d-4c6e-98b6-9ebc465096f4-data-volume\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:25:59.970125 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:25:59.970103    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plss\" (UniqueName: \"kubernetes.io/projected/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-api-access-2plss\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:00.245805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.245776    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:26:00.246365 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.246347    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/0.log"
Jul 24 14:26:00.246451 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.246385    2584 generic.go:356] "Generic (PLEG): container finished" podID="74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7" containerID="2e0eb4ce3e56d30bc38e97f1f41370c2896b741e35a0562a316ecb988e4574fa" exitCode=255
Jul 24 14:26:00.246566 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.246527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" event={"ID":"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7","Type":"ContainerDied","Data":"2e0eb4ce3e56d30bc38e97f1f41370c2896b741e35a0562a316ecb988e4574fa"}
Jul 24 14:26:00.246686 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.246622    2584 scope.go:117] "RemoveContainer" containerID="710079a6ed9715309b8d786bc507492f904db41000cac1430fa8e6939cc8ec9d"
Jul 24 14:26:00.246778 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.246754    2584 scope.go:117] "RemoveContainer" containerID="2e0eb4ce3e56d30bc38e97f1f41370c2896b741e35a0562a316ecb988e4574fa"
Jul 24 14:26:00.247013 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.246990    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fxg9p_openshift-console-operator(74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" podUID="74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7"
Jul 24 14:26:00.463406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.463378    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:00.463572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.463440    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:00.463572 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.463493    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:26:00.463572 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.463538    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:00.463693 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.463577    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:01.463561449 +0000 UTC m=+218.248261892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : configmap "kube-rbac-proxy" not found
Jul 24 14:26:00.463693 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.463592    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:01.463585755 +0000 UTC m=+218.248286197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:00.968504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.968468    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:00.968683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.968563    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:26:00.968683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.968581    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:26:00.968683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.968608    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:00.968683 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968617    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.968626    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968692    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968702    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:26:16.968678492 +0000 UTC m=+233.753378949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : secret "router-metrics-certs-default" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968719    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968693    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968742    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle podName:6eeb8c69-5d01-44e5-9fa3-77a9687c007b nodeName:}" failed. No retries permitted until 2026-07-24 14:26:16.968729998 +0000 UTC m=+233.753430455 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle") pod "router-default-df9867497-4w8nq" (UID: "6eeb8c69-5d01-44e5-9fa3-77a9687c007b") : configmap references non-existent config key: service-ca.crt
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968739    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6754854769-k8g7q: secret "image-registry-tls" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:00.968770    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968815    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls podName:c4f73506-ad36-43a8-8685-b5daf2987c1f nodeName:}" failed. No retries permitted until 2026-07-24 14:26:16.968797852 +0000 UTC m=+233.753498318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls") pod "image-registry-6754854769-k8g7q" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f") : secret "image-registry-tls" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968823    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968864    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls podName:6852cf8e-b13e-49e3-b9bb-383a5ea7e803 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:16.968853852 +0000 UTC m=+233.753554313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kschn" (UID: "6852cf8e-b13e-49e3-b9bb-383a5ea7e803") : secret "cluster-monitoring-operator-tls" not found
Jul 24 14:26:00.968884 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968885    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls podName:a64d93b6-1a8a-4094-9e1f-d2449c1bcf67 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:16.968876796 +0000 UTC m=+233.753577242 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls") pod "cluster-samples-operator-6748489656-nrlqz" (UID: "a64d93b6-1a8a-4094-9e1f-d2449c1bcf67") : secret "samples-operator-tls" not found
Jul 24 14:26:00.969252 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:00.968901    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert podName:d0522bad-1a97-410d-bbb2-0037cdc9852d nodeName:}" failed. No retries permitted until 2026-07-24 14:26:16.968895393 +0000 UTC m=+233.753595835 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-k6dbg" (UID: "d0522bad-1a97-410d-bbb2-0037cdc9852d") : secret "networking-console-plugin-cert" not found
Jul 24 14:26:01.001131 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.001111    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2htf_be0981fc-a012-4c1d-8282-653973ba75b5/dns-node-resolver/0.log"
Jul 24 14:26:01.069682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.069657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:26:01.069848 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.069817    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 14:26:01.069904 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.069885    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls podName:c872d451-23b6-49e7-b8b0-a41cdcbb4ff5 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:17.069870089 +0000 UTC m=+233.854570553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls") pod "dns-default-ssbjq" (UID: "c872d451-23b6-49e7-b8b0-a41cdcbb4ff5") : secret "dns-default-metrics-tls" not found
Jul 24 14:26:01.251226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.251153    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:26:01.251611 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.251460    2584 scope.go:117] "RemoveContainer" containerID="2e0eb4ce3e56d30bc38e97f1f41370c2896b741e35a0562a316ecb988e4574fa"
Jul 24 14:26:01.251657 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.251613    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fxg9p_openshift-console-operator(74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" podUID="74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7"
Jul 24 14:26:01.448869 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.448824    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"]
Jul 24 14:26:01.453267 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.453249    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"
Jul 24 14:26:01.457759 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.457734    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 14:26:01.458182 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.458162    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b4jzt\""
Jul 24 14:26:01.458266 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.458221    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 14:26:01.468886 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.468861    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"]
Jul 24 14:26:01.473460 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.473432    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:01.473564 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.473535    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:01.473621 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.473586    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:03.473572875 +0000 UTC m=+220.258273319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:01.473673 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.473641    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:01.473758 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.473744    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:26:01.473801 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:01.473784    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:03.473775426 +0000 UTC m=+220.258475869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : configmap "kube-rbac-proxy" not found
Jul 24 14:26:01.574529 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.574500    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8jtm\" (UniqueName: \"kubernetes.io/projected/d66d09b5-1150-470e-818e-1eed339010cb-kube-api-access-g8jtm\") pod \"migrator-6865c9967-fgg7k\" (UID: \"d66d09b5-1150-470e-818e-1eed339010cb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"
Jul 24 14:26:01.675811 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.675789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8jtm\" (UniqueName: \"kubernetes.io/projected/d66d09b5-1150-470e-818e-1eed339010cb-kube-api-access-g8jtm\") pod \"migrator-6865c9967-fgg7k\" (UID: \"d66d09b5-1150-470e-818e-1eed339010cb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"
Jul 24 14:26:01.686403 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.686378    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8jtm\" (UniqueName: \"kubernetes.io/projected/d66d09b5-1150-470e-818e-1eed339010cb-kube-api-access-g8jtm\") pod \"migrator-6865c9967-fgg7k\" (UID: \"d66d09b5-1150-470e-818e-1eed339010cb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"
Jul 24 14:26:01.762810 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.762789    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"
Jul 24 14:26:01.898583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:01.898558    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k"]
Jul 24 14:26:01.899704 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:01.899674    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66d09b5_1150_470e_818e_1eed339010cb.slice/crio-cfe95b1a2e170560430a5b30ae1ed8655148c1539810a2cc1265442fa7ecdfc2 WatchSource:0}: Error finding container cfe95b1a2e170560430a5b30ae1ed8655148c1539810a2cc1265442fa7ecdfc2: Status 404 returned error can't find the container with id cfe95b1a2e170560430a5b30ae1ed8655148c1539810a2cc1265442fa7ecdfc2
Jul 24 14:26:02.001769 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.001746    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hpn4m_736d187e-5b4a-41b9-a918-a723cec1dcd7/node-ca/0.log"
Jul 24 14:26:02.150556 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.150487    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-cgcgg"]
Jul 24 14:26:02.154795 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.154779    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.159207 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.159183    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 14:26:02.159300 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.159248    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 14:26:02.160158 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.160141    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bbvvg\""
Jul 24 14:26:02.160787 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.160772    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 14:26:02.160943 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.160926    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 14:26:02.163335 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.163317    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-cgcgg"]
Jul 24 14:26:02.255475 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.255452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k" event={"ID":"d66d09b5-1150-470e-818e-1eed339010cb","Type":"ContainerStarted","Data":"cfe95b1a2e170560430a5b30ae1ed8655148c1539810a2cc1265442fa7ecdfc2"}
Jul 24 14:26:02.282891 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.282862    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-signing-key\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.283004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.282935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-signing-cabundle\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.283067 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.283054    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzmdr\" (UniqueName: \"kubernetes.io/projected/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-kube-api-access-tzmdr\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.383545 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.383517    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzmdr\" (UniqueName: \"kubernetes.io/projected/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-kube-api-access-tzmdr\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.383666 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.383582    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-signing-key\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.383784 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.383758    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-signing-cabundle\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.384383 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.384361    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-signing-cabundle\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.385955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.385934    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-signing-key\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.393466 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.393443    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzmdr\" (UniqueName: \"kubernetes.io/projected/5a8c0410-34a4-44fe-8de0-1c6f38b741aa-kube-api-access-tzmdr\") pod \"service-ca-658d7b7c78-cgcgg\" (UID: \"5a8c0410-34a4-44fe-8de0-1c6f38b741aa\") " pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.464255 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.464230    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg"
Jul 24 14:26:02.602676 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.602652    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbkpq_ea5c9656-f807-497c-8f41-715f74b4385f/serve-healthcheck-canary/0.log"
Jul 24 14:26:02.603170 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:02.603149    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-cgcgg"]
Jul 24 14:26:02.605435 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:02.605407    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8c0410_34a4_44fe_8de0_1c6f38b741aa.slice/crio-fc1c2aa600bdcb86a9dc1ba622b565900700214e3bf572b389bf6158e79d8678 WatchSource:0}: Error finding container fc1c2aa600bdcb86a9dc1ba622b565900700214e3bf572b389bf6158e79d8678: Status 404 returned error can't find the container with id fc1c2aa600bdcb86a9dc1ba622b565900700214e3bf572b389bf6158e79d8678
Jul 24 14:26:03.260772 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.260733    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k" event={"ID":"d66d09b5-1150-470e-818e-1eed339010cb","Type":"ContainerStarted","Data":"a04c2d95e3298c2d7518c2c7eb8e35953dc1ec2497c5249784dda944618759f6"}
Jul 24 14:26:03.261204 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.260796    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k" event={"ID":"d66d09b5-1150-470e-818e-1eed339010cb","Type":"ContainerStarted","Data":"5d294fa3ec4ae6c81b56feb5b960e8378607b193bcb6b1498b77acd64cf3744d"}
Jul 24 14:26:03.262139 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.262119    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg" event={"ID":"5a8c0410-34a4-44fe-8de0-1c6f38b741aa","Type":"ContainerStarted","Data":"475959a1e79e8ddd1582bc68fc2fb188feeca5ce43c01156d13867e9f381e40f"}
Jul 24 14:26:03.262228 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.262154    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg" event={"ID":"5a8c0410-34a4-44fe-8de0-1c6f38b741aa","Type":"ContainerStarted","Data":"fc1c2aa600bdcb86a9dc1ba622b565900700214e3bf572b389bf6158e79d8678"}
Jul 24 14:26:03.282946 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.282856    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fgg7k" podStartSLOduration=1.151746553 podStartE2EDuration="2.282816413s" totalImagesPullingTime="1.13106986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:03.281499481 +0000 UTC m=+220.066199944" watchObservedRunningTime="2026-07-24 14:26:03.282816413 +0000 UTC m=+220.067516881"
Jul 24 14:26:03.302056 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.302008    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-cgcgg" podStartSLOduration=1.301992663 podStartE2EDuration="1.301992663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:03.300439098 +0000 UTC m=+220.085139563" watchObservedRunningTime="2026-07-24 14:26:03.301992663 +0000 UTC m=+220.086693123"
Jul 24 14:26:03.495338 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.495305    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:03.495530 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:03.495369    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:03.495530 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:03.495437    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:26:03.495530 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:03.495469    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:03.495530 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:03.495513    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:07.495492927 +0000 UTC m=+224.280193386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : configmap "kube-rbac-proxy" not found
Jul 24 14:26:03.495530 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:03.495533    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:07.495522603 +0000 UTC m=+224.280223061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:05.477897 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:05.477862    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:26:05.478243 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:05.477913    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:26:05.478243 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:05.478222    2584 scope.go:117] "RemoveContainer" containerID="2e0eb4ce3e56d30bc38e97f1f41370c2896b741e35a0562a316ecb988e4574fa"
Jul 24 14:26:05.478388 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:05.478370    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fxg9p_openshift-console-operator(74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" podUID="74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7"
Jul 24 14:26:06.106281 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:06.106253    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f9csd"
Jul 24 14:26:07.535150 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:07.535113    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:07.535547 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:07.535247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:07.535547 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:07.535260    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:07.535547 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:07.535325    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 14:26:07.535547 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:07.535328    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:15.535311246 +0000 UTC m=+232.320011691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : secret "insights-runtime-extractor-tls" not found
Jul 24 14:26:07.535547 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:07.535393    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm podName:39a5116a-222d-4c6e-98b6-9ebc465096f4 nodeName:}" failed. No retries permitted until 2026-07-24 14:26:15.5353751 +0000 UTC m=+232.320075552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t6jpv" (UID: "39a5116a-222d-4c6e-98b6-9ebc465096f4") : configmap "kube-rbac-proxy" not found
Jul 24 14:26:13.294394 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.294367    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_74ca4cffd660659a68e7ccdd89a85ecc/setup/4.log"
Jul 24 14:26:13.294928 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.294402    2584 generic.go:356] "Generic (PLEG): container finished" podID="74ca4cffd660659a68e7ccdd89a85ecc" containerID="4d7efe6f5f02b33da78f66ac401602074b60f5acac5e61ff9c4b734169f3042e" exitCode=0
Jul 24 14:26:13.294928 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.294462    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerDied","Data":"4d7efe6f5f02b33da78f66ac401602074b60f5acac5e61ff9c4b734169f3042e"}
Jul 24 14:26:13.294928 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.294515    2584 scope.go:117] "RemoveContainer" containerID="a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5"
Jul 24 14:26:13.294928 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.294878    2584 scope.go:117] "RemoveContainer" containerID="a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5"
Jul 24 14:26:13.321012 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:13.320984    2584 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_4 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5'" containerID="a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5"
Jul 24 14:26:13.321143 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:26:13.321031    2584 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator_74ca4cffd660659a68e7ccdd89a85ecc_4 in pod sandbox c9e7045985861a92207da4e0c1e4792fb1a3dcea510715e22112c0475c2a9fdc from index: no such id: 'a9cf5a935c9a28428753d84d0737f3def7a6bcac44e54985c3636e93aab2b5e5'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal_openshift-machine-config-operator(74ca4cffd660659a68e7ccdd89a85ecc)\"" logger="UnhandledError"
Jul 24 14:26:13.390111 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.390006    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:26:13.390239 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.390165    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:26:13.392532 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.392509    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7478e2db-f305-4c13-8f1b-2e881537d6a4-metrics-certs\") pod \"network-metrics-daemon-4gdl8\" (UID: \"7478e2db-f305-4c13-8f1b-2e881537d6a4\") " pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:26:13.392616 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.392600    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d36989a-3602-4e05-8ab1-7f85407fd61c-original-pull-secret\") pod \"global-pull-secret-syncer-wz7zb\" (UID: \"8d36989a-3602-4e05-8ab1-7f85407fd61c\") " pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:26:13.480455 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.480434    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gdl8"
Jul 24 14:26:13.485007 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.484982    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wz7zb"
Jul 24 14:26:13.491369 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.491347    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:26:13.493793 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.493772    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbnm2\" (UniqueName: \"kubernetes.io/projected/d9e1f31f-b9b2-418a-8d01-399553e9f5a8-kube-api-access-qbnm2\") pod \"network-check-target-jfv8z\" (UID: \"d9e1f31f-b9b2-418a-8d01-399553e9f5a8\") " pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:26:13.641767 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.641737    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4gdl8"]
Jul 24 14:26:13.644466 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:13.644440    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7478e2db_f305_4c13_8f1b_2e881537d6a4.slice/crio-4a37fea8f01ef17ff82601a753859980490562ba0f28ee8ecfd7dfa9947663ae WatchSource:0}: Error finding container 4a37fea8f01ef17ff82601a753859980490562ba0f28ee8ecfd7dfa9947663ae: Status 404 returned error can't find the container with id 4a37fea8f01ef17ff82601a753859980490562ba0f28ee8ecfd7dfa9947663ae
Jul 24 14:26:13.654687 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.654662    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wz7zb"]
Jul 24 14:26:13.657051 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:13.657027    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d36989a_3602_4e05_8ab1_7f85407fd61c.slice/crio-7d3c4aa6e6e3fe34481b88006aef4f4b238a0f69dcd60940c809e6ce94092ad7 WatchSource:0}: Error finding container 7d3c4aa6e6e3fe34481b88006aef4f4b238a0f69dcd60940c809e6ce94092ad7: Status 404 returned error can't find the container with id 7d3c4aa6e6e3fe34481b88006aef4f4b238a0f69dcd60940c809e6ce94092ad7
Jul 24 14:26:13.774737 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.774711    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:26:13.923548 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:13.923181    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jfv8z"]
Jul 24 14:26:13.926131 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:13.926103    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e1f31f_b9b2_418a_8d01_399553e9f5a8.slice/crio-91ec183abd89f72c967a4eb7bb53cfdd3d694f88ef39c12a46e38c9138ee3a9b WatchSource:0}: Error finding container 91ec183abd89f72c967a4eb7bb53cfdd3d694f88ef39c12a46e38c9138ee3a9b: Status 404 returned error can't find the container with id 91ec183abd89f72c967a4eb7bb53cfdd3d694f88ef39c12a46e38c9138ee3a9b
Jul 24 14:26:14.301059 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.301014    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" event={"ID":"74ca4cffd660659a68e7ccdd89a85ecc","Type":"ContainerStarted","Data":"8a1016ad8060390e4bc2877ed79a8dd0d5c36ef4e7e0578747893a0f8307467e"}
Jul 24 14:26:14.302345 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.302311    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wz7zb" event={"ID":"8d36989a-3602-4e05-8ab1-7f85407fd61c","Type":"ContainerStarted","Data":"7d3c4aa6e6e3fe34481b88006aef4f4b238a0f69dcd60940c809e6ce94092ad7"}
Jul 24 14:26:14.303604 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.303580    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gdl8" event={"ID":"7478e2db-f305-4c13-8f1b-2e881537d6a4","Type":"ContainerStarted","Data":"4a37fea8f01ef17ff82601a753859980490562ba0f28ee8ecfd7dfa9947663ae"}
Jul 24 14:26:14.305303 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.305270    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jfv8z" event={"ID":"d9e1f31f-b9b2-418a-8d01-399553e9f5a8","Type":"ContainerStarted","Data":"11937a23b9286bad1ffd8a206b24ca8bc2a68b6aaa785f5529f6ec62a2666dc9"}
Jul 24 14:26:14.305419 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.305307    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jfv8z" event={"ID":"d9e1f31f-b9b2-418a-8d01-399553e9f5a8","Type":"ContainerStarted","Data":"91ec183abd89f72c967a4eb7bb53cfdd3d694f88ef39c12a46e38c9138ee3a9b"}
Jul 24 14:26:14.305419 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.305384    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:26:14.323442 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.323392    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-93.ec2.internal" podStartSLOduration=65.323377675 podStartE2EDuration="1m5.323377675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:14.321970337 +0000 UTC m=+231.106670801" watchObservedRunningTime="2026-07-24 14:26:14.323377675 +0000 UTC m=+231.108078137"
Jul 24 14:26:14.350976 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:14.350929    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jfv8z" podStartSLOduration=71.350909743 podStartE2EDuration="1m11.350909743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:14.34946413 +0000 UTC m=+231.134164605" watchObservedRunningTime="2026-07-24 14:26:14.350909743 +0000 UTC m=+231.135610207"
Jul 24 14:26:15.311376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.311336    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gdl8" event={"ID":"7478e2db-f305-4c13-8f1b-2e881537d6a4","Type":"ContainerStarted","Data":"9a9cf00015895cc9f00289d6f3aa361be81a5acc058a7a517fafcc636fa96cd2"}
Jul 24 14:26:15.311824 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.311434    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gdl8" event={"ID":"7478e2db-f305-4c13-8f1b-2e881537d6a4","Type":"ContainerStarted","Data":"21b1cd19502d2b9f58ffa0605bc6b55e9b133cf382cc1265dc56b55d783b5945"}
Jul 24 14:26:15.335951 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.335897    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4gdl8" podStartSLOduration=70.001457692 podStartE2EDuration="1m11.335876884s" totalImagesPullingTime="1.334419192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:15.335780885 +0000 UTC m=+232.120481340" watchObservedRunningTime="2026-07-24 14:26:15.335876884 +0000 UTC m=+232.120577349"
Jul 24 14:26:15.612166 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.612087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:15.612311 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.612198    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:15.612656 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.612635    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/39a5116a-222d-4c6e-98b6-9ebc465096f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:15.614355 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.614331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/39a5116a-222d-4c6e-98b6-9ebc465096f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t6jpv\" (UID: \"39a5116a-222d-4c6e-98b6-9ebc465096f4\") " pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:15.670256 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.670232    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t6jpv"
Jul 24 14:26:15.814349 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:15.814316    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t6jpv"]
Jul 24 14:26:15.817656 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:15.817630    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a5116a_222d_4c6e_98b6_9ebc465096f4.slice/crio-dc156ee9fbc0ca465856e7eaf583fba099c0fef85ed911ddbe2f2fe2189ee650 WatchSource:0}: Error finding container dc156ee9fbc0ca465856e7eaf583fba099c0fef85ed911ddbe2f2fe2189ee650: Status 404 returned error can't find the container with id dc156ee9fbc0ca465856e7eaf583fba099c0fef85ed911ddbe2f2fe2189ee650
Jul 24 14:26:16.317068 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:16.317035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6jpv" event={"ID":"39a5116a-222d-4c6e-98b6-9ebc465096f4","Type":"ContainerStarted","Data":"dc156ee9fbc0ca465856e7eaf583fba099c0fef85ed911ddbe2f2fe2189ee650"}
Jul 24 14:26:17.024691 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.024639    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:17.024908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.024723    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:26:17.024908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.024749    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:26:17.024908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.024785    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:17.024908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.024808    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:26:17.024908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.024854    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:26:17.026257 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.026206    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-service-ca-bundle\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:17.028483 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.028454    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb8c69-5d01-44e5-9fa3-77a9687c007b-metrics-certs\") pod \"router-default-df9867497-4w8nq\" (UID: \"6eeb8c69-5d01-44e5-9fa3-77a9687c007b\") " pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:17.028600 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.028490    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a64d93b6-1a8a-4094-9e1f-d2449c1bcf67-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-nrlqz\" (UID: \"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:26:17.028600 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.028574    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6852cf8e-b13e-49e3-b9bb-383a5ea7e803-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kschn\" (UID: \"6852cf8e-b13e-49e3-b9bb-383a5ea7e803\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:26:17.029766 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.029689    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"image-registry-6754854769-k8g7q\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") " pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:26:17.030193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.030164    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d0522bad-1a97-410d-bbb2-0037cdc9852d-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-k6dbg\" (UID: \"d0522bad-1a97-410d-bbb2-0037cdc9852d\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:26:17.126162 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.126119    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:26:17.129073 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.129019    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c872d451-23b6-49e7-b8b0-a41cdcbb4ff5-metrics-tls\") pod \"dns-default-ssbjq\" (UID: \"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5\") " pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:26:17.133960 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.133888    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"
Jul 24 14:26:17.144027 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.143975    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"
Jul 24 14:26:17.161422 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.161395    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:26:17.171316 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.171297    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:17.196159 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.196139    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"
Jul 24 14:26:17.290996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:17.290914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:26:18.476337 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.476229    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6754854769-k8g7q"]
Jul 24 14:26:18.481453 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.481353    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg"]
Jul 24 14:26:18.482608 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:18.482567    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f73506_ad36_43a8_8685_b5daf2987c1f.slice/crio-813ac890aeed92cba2ab978edc483ad319c23339ce6ba72b761e58103a23677d WatchSource:0}: Error finding container 813ac890aeed92cba2ab978edc483ad319c23339ce6ba72b761e58103a23677d: Status 404 returned error can't find the container with id 813ac890aeed92cba2ab978edc483ad319c23339ce6ba72b761e58103a23677d
Jul 24 14:26:18.490758 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:18.490721    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0522bad_1a97_410d_bbb2_0037cdc9852d.slice/crio-db42630508aac3160c52f078cb82ca982405f174d40af30684fb75bb3e266f45 WatchSource:0}: Error finding container db42630508aac3160c52f078cb82ca982405f174d40af30684fb75bb3e266f45: Status 404 returned error can't find the container with id db42630508aac3160c52f078cb82ca982405f174d40af30684fb75bb3e266f45
Jul 24 14:26:18.516073 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.516045    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-df9867497-4w8nq"]
Jul 24 14:26:18.527300 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:18.526586    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eeb8c69_5d01_44e5_9fa3_77a9687c007b.slice/crio-d9f99575d27a4c106cb731bfaaf130d6a7cc3537b4b541772a2744830e6f04cb WatchSource:0}: Error finding container d9f99575d27a4c106cb731bfaaf130d6a7cc3537b4b541772a2744830e6f04cb: Status 404 returned error can't find the container with id d9f99575d27a4c106cb731bfaaf130d6a7cc3537b4b541772a2744830e6f04cb
Jul 24 14:26:18.540108 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.539559    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz"]
Jul 24 14:26:18.556372 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.556309    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn"]
Jul 24 14:26:18.558042 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.558015    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ssbjq"]
Jul 24 14:26:18.665183 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:18.665155    2584 scope.go:117] "RemoveContainer" containerID="2e0eb4ce3e56d30bc38e97f1f41370c2896b741e35a0562a316ecb988e4574fa"
Jul 24 14:26:19.333213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.333183    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg" event={"ID":"d0522bad-1a97-410d-bbb2-0037cdc9852d","Type":"ContainerStarted","Data":"db42630508aac3160c52f078cb82ca982405f174d40af30684fb75bb3e266f45"}
Jul 24 14:26:19.334759 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.334729    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wz7zb" event={"ID":"8d36989a-3602-4e05-8ab1-7f85407fd61c","Type":"ContainerStarted","Data":"303939f878643fec2fc66d41738a972561e108b18d35cf95c5d07791a173b7cb"}
Jul 24 14:26:19.336390 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.336361    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6754854769-k8g7q" event={"ID":"c4f73506-ad36-43a8-8685-b5daf2987c1f","Type":"ContainerStarted","Data":"20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c"}
Jul 24 14:26:19.336491 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.336412    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6754854769-k8g7q" event={"ID":"c4f73506-ad36-43a8-8685-b5daf2987c1f","Type":"ContainerStarted","Data":"813ac890aeed92cba2ab978edc483ad319c23339ce6ba72b761e58103a23677d"}
Jul 24 14:26:19.336491 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.336437    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:26:19.337629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.337602    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-df9867497-4w8nq" event={"ID":"6eeb8c69-5d01-44e5-9fa3-77a9687c007b","Type":"ContainerStarted","Data":"d9f99575d27a4c106cb731bfaaf130d6a7cc3537b4b541772a2744830e6f04cb"}
Jul 24 14:26:19.363925 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.363881    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wz7zb" podStartSLOduration=70.738994736 podStartE2EDuration="1m15.36386768s" totalImagesPullingTime="4.624872944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:19.362333464 +0000 UTC m=+236.147033929" watchObservedRunningTime="2026-07-24 14:26:19.36386768 +0000 UTC m=+236.148568143"
Jul 24 14:26:19.392079 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:19.392037    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6754854769-k8g7q" podStartSLOduration=78.392019447 podStartE2EDuration="1m18.392019447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:19.391954426 +0000 UTC m=+236.176654892" watchObservedRunningTime="2026-07-24 14:26:19.392019447 +0000 UTC m=+236.176719916"
Jul 24 14:26:20.576945 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:20.576908    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6852cf8e_b13e_49e3_b9bb_383a5ea7e803.slice/crio-a6635c78342b47c02e44389eb68467bdcb1b3e7deddb4f20bd150de0227169c0 WatchSource:0}: Error finding container a6635c78342b47c02e44389eb68467bdcb1b3e7deddb4f20bd150de0227169c0: Status 404 returned error can't find the container with id a6635c78342b47c02e44389eb68467bdcb1b3e7deddb4f20bd150de0227169c0
Jul 24 14:26:20.577797 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:20.577663    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc872d451_23b6_49e7_b8b0_a41cdcbb4ff5.slice/crio-76ec7a16956173d1bea9e74c10468b0696f3dcf055f1dfd302a4dc7f017ac41f WatchSource:0}: Error finding container 76ec7a16956173d1bea9e74c10468b0696f3dcf055f1dfd302a4dc7f017ac41f: Status 404 returned error can't find the container with id 76ec7a16956173d1bea9e74c10468b0696f3dcf055f1dfd302a4dc7f017ac41f
Jul 24 14:26:21.347988 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.347950    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ssbjq" event={"ID":"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5","Type":"ContainerStarted","Data":"76ec7a16956173d1bea9e74c10468b0696f3dcf055f1dfd302a4dc7f017ac41f"}
Jul 24 14:26:21.349226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.349195    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn" event={"ID":"6852cf8e-b13e-49e3-b9bb-383a5ea7e803","Type":"ContainerStarted","Data":"a6635c78342b47c02e44389eb68467bdcb1b3e7deddb4f20bd150de0227169c0"}
Jul 24 14:26:21.351290 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.351264    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-df9867497-4w8nq" event={"ID":"6eeb8c69-5d01-44e5-9fa3-77a9687c007b","Type":"ContainerStarted","Data":"4867ccab6f16480055e6f6d19b69d870e9a3470670b50951038ff00ba08999a8"}
Jul 24 14:26:21.354579 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.354554    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:26:21.354699 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.354618    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" event={"ID":"74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7","Type":"ContainerStarted","Data":"e7699102995aa8d92a2a61e73670c8c04cc0f354a4a52ab71a48d0bd2b3b12ee"}
Jul 24 14:26:21.355058 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.355039    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:26:21.356197 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.356140    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz" event={"ID":"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67","Type":"ContainerStarted","Data":"ce3c11d00578fbf82b22066579a8ea352d6dbd32588e60dbec971ecbc43745f5"}
Jul 24 14:26:21.356444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.356358    2584 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-fxg9p container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.134.0.18:8443/readyz\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body=
Jul 24 14:26:21.356444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.356404    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" podUID="74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7" containerName="console-operator" probeResult="failure" output="Get \"https://10.134.0.18:8443/readyz\": dial tcp 10.134.0.18:8443: connect: connection refused"
Jul 24 14:26:21.380760 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.380699    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-df9867497-4w8nq" podStartSLOduration=80.380680097 podStartE2EDuration="1m20.380680097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:21.380435415 +0000 UTC m=+238.165135904" watchObservedRunningTime="2026-07-24 14:26:21.380680097 +0000 UTC m=+238.165380562"
Jul 24 14:26:21.401641 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:21.401589    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p" podStartSLOduration=66.836080598 podStartE2EDuration="1m19.401569595s" totalImagesPullingTime="12.565488997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:21.40157068 +0000 UTC m=+238.186271144" watchObservedRunningTime="2026-07-24 14:26:21.401569595 +0000 UTC m=+238.186270060"
Jul 24 14:26:22.172299 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.172269    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:22.175135 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.175102    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:22.175135 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:22.175135 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:22.175135 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:22.175475 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.175163    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:22.365323 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.365226    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6jpv" event={"ID":"39a5116a-222d-4c6e-98b6-9ebc465096f4","Type":"ContainerStarted","Data":"28042fcdc21393f535217ac396161cde9c6b3c89bba34b9cd1c68645e8aa4790"}
Jul 24 14:26:22.370979 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.370594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg" event={"ID":"d0522bad-1a97-410d-bbb2-0037cdc9852d","Type":"ContainerStarted","Data":"95db08e51c510d45e90cff107287e4602b7decb58fe8155f61324584f320b93b"}
Jul 24 14:26:22.377350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.377293    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-fxg9p"
Jul 24 14:26:22.396676 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:22.395916    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-k6dbg" podStartSLOduration=77.177575383 podStartE2EDuration="1m20.395896381s" totalImagesPullingTime="3.218320998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:22.394672001 +0000 UTC m=+239.179372465" watchObservedRunningTime="2026-07-24 14:26:22.395896381 +0000 UTC m=+239.180596848"
Jul 24 14:26:23.173499 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:23.173462    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:23.173499 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:23.173499 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:23.173499 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:23.174241 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:23.173520    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:24.173048 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:24.173002    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:24.173048 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:24.173048 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:24.173048 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:24.173369 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:24.173074    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:25.173677 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:25.173639    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:25.173677 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:25.173677 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:25.173677 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:25.174218 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:25.173702    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:26.173261 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:26.173225    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:26.173261 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:26.173261 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:26.173261 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:26.173518 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:26.173290    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:27.172345 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.172308    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:27.173469 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.173438    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:27.173469 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:27.173469 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:27.173469 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:27.173727 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.173500    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:27.391485 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.391427    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ssbjq" event={"ID":"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5","Type":"ContainerStarted","Data":"402812d18438614c1478dc76f90029d647acbeba5f5a2395ed62f5373750340f"}
Jul 24 14:26:27.391485 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.391491    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ssbjq" event={"ID":"c872d451-23b6-49e7-b8b0-a41cdcbb4ff5","Type":"ContainerStarted","Data":"d16909ace420719c38762c9a79b5d04e531fb505b4e561bb5c991240186512f6"}
Jul 24 14:26:27.391739 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.391624    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:26:27.393246 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.393199    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn" event={"ID":"6852cf8e-b13e-49e3-b9bb-383a5ea7e803","Type":"ContainerStarted","Data":"6724e2cae3f704a6815e79198475748ea83dfee853b252c965f11c21f732fd5a"}
Jul 24 14:26:27.395421 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.395282    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz" event={"ID":"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67","Type":"ContainerStarted","Data":"b10dbc54f0159a553143cac257bb52141437df8e2887dd74163231ac2a38e183"}
Jul 24 14:26:27.395421 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.395332    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz" event={"ID":"a64d93b6-1a8a-4094-9e1f-d2449c1bcf67","Type":"ContainerStarted","Data":"47bac7973b02f1a0a9641358cd2a45ee5f0e125044956f57971db23be5af4b84"}
Jul 24 14:26:27.397227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.397198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6jpv" event={"ID":"39a5116a-222d-4c6e-98b6-9ebc465096f4","Type":"ContainerStarted","Data":"826f37c841df503f0038c7c36ad5e4c4be0aabf1672d40c24c84ac495f6a3e12"}
Jul 24 14:26:27.422018 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.421972    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ssbjq" podStartSLOduration=36.720473402 podStartE2EDuration="42.421954829s" totalImagesPullingTime="5.701481427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:27.419100243 +0000 UTC m=+244.203800707" watchObservedRunningTime="2026-07-24 14:26:27.421954829 +0000 UTC m=+244.206655297"
Jul 24 14:26:27.443345 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.443259    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kschn" podStartSLOduration=80.740408562 podStartE2EDuration="1m26.443241512s" totalImagesPullingTime="5.70283295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:27.44300723 +0000 UTC m=+244.227707708" watchObservedRunningTime="2026-07-24 14:26:27.443241512 +0000 UTC m=+244.227941977"
Jul 24 14:26:27.463150 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:27.463104    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-nrlqz" podStartSLOduration=80.809398024 podStartE2EDuration="1m26.463091236s" totalImagesPullingTime="5.653693212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:27.462698132 +0000 UTC m=+244.247398597" watchObservedRunningTime="2026-07-24 14:26:27.463091236 +0000 UTC m=+244.247791700"
Jul 24 14:26:28.172854 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:28.172794    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:28.172854 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:28.172854 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:28.172854 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:28.173391 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:28.172880    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:29.173384 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:29.173346    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:29.173384 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:29.173384 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:29.173384 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:29.173869 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:29.173417    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:30.172873 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:30.172819    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:30.172873 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:30.172873 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:30.172873 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:30.173128 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:30.172896    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:31.172717 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:31.172645    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:31.172717 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:31.172717 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:31.172717 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:31.172717 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:31.172694    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:31.415468 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:31.415431    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t6jpv" event={"ID":"39a5116a-222d-4c6e-98b6-9ebc465096f4","Type":"ContainerStarted","Data":"a940503d0e9443150054d2cbe602e173dc5644e9011dc0686a2663aaa9b6270a"}
Jul 24 14:26:31.443137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:31.443038    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t6jpv" podStartSLOduration=17.668767312 podStartE2EDuration="32.44302239s" totalImagesPullingTime="14.774255078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:25:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:31.441429877 +0000 UTC m=+248.226130340" watchObservedRunningTime="2026-07-24 14:26:31.44302239 +0000 UTC m=+248.227722853"
Jul 24 14:26:32.172546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:32.172514    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:32.172546 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:32.172546 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:32.172546 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:32.173027 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:32.172566    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:33.172529 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:33.172486    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:33.172529 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:33.172529 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:33.172529 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:33.173159 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:33.172541    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:34.172922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:34.172880    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:34.172922 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:34.172922 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:34.172922 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:34.173340 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:34.172938    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:35.172924 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:35.172888    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:35.172924 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:35.172924 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:35.172924 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:35.173363 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:35.172945    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:36.173532 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:36.173491    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:36.173532 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:36.173532 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:36.173532 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:36.174053 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:36.173549    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:37.167064 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:37.167034    2584 patch_prober.go:28] interesting pod/image-registry-6754854769-k8g7q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 14:26:37.167225 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:37.167080    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6754854769-k8g7q" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:26:37.172979 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:37.172948    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:37.172979 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:37.172979 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:37.172979 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:37.173197 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:37.172994    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:37.403675 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:37.403649    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ssbjq"
Jul 24 14:26:38.172777 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:38.172743    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:38.172777 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:38.172777 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:38.172777 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:38.173111 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:38.172800    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:39.173107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:39.173075    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:39.173107 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:39.173107 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:39.173107 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:39.173541 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:39.173125    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:40.173632 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:40.173601    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:40.173632 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:40.173632 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:40.173632 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:40.174080 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:40.173654    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:40.345387 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:40.345358    2584 patch_prober.go:28] interesting pod/image-registry-6754854769-k8g7q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 14:26:40.345562 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:40.345404    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6754854769-k8g7q" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:26:41.173380 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:41.173344    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:41.173380 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:41.173380 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:41.173380 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:41.173623 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:41.173398    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:42.173025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:42.172988    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:42.173025 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:42.173025 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:42.173025 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:42.173454 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:42.173042    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:43.173222 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:43.173192    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:43.173222 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:43.173222 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:43.173222 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:43.173634 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:43.173243    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:44.173273 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:44.173240    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:44.173273 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:44.173273 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:44.173273 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:44.173713 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:44.173290    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:45.173230 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:45.173190    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:45.173230 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:45.173230 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:45.173230 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:45.173760 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:45.173264    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:45.313966 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:45.313938    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jfv8z"
Jul 24 14:26:46.172982 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:46.172945    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:46.172982 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:46.172982 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:46.172982 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:46.173216 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:46.172998    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:47.165932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:47.165895    2584 patch_prober.go:28] interesting pod/image-registry-6754854769-k8g7q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 14:26:47.166316 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:47.165946    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6754854769-k8g7q" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:26:47.172874 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:47.172828    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:47.172874 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:47.172874 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:47.172874 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:47.173074 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:47.172891    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:48.173123 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:48.173091    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:48.173123 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:48.173123 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:48.173123 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:48.173578 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:48.173143    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:49.173508 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:49.173475    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:49.173508 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:49.173508 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:49.173508 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:49.174085 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:49.173533    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:50.172471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:50.172435    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:50.172471 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:50.172471 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:50.172471 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:50.172710 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:50.172487    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:50.345870 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:50.345824    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:26:51.173021 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:51.172991    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:51.173021 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:51.173021 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:51.173021 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:51.173260 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:51.173042    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:52.172964 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:52.172929    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:52.172964 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:52.172964 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:52.172964 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:52.173406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:52.172987    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:53.173232 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:53.173198    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:53.173232 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:53.173232 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:53.173232 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:53.173646 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:53.173251    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:54.173360 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:54.173325    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:54.173360 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:54.173360 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:54.173360 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:54.173878 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:54.173385    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:55.173455 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:55.173417    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:55.173455 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:55.173455 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:55.173455 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:55.173925 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:55.173481    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:56.173491 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.173457    2584 patch_prober.go:28] interesting pod/router-default-df9867497-4w8nq container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Jul 24 14:26:56.173491 ip-10-0-135-93 kubenswrapper[2584]: [-]has-synced failed: reason withheld
Jul 24 14:26:56.173491 ip-10-0-135-93 kubenswrapper[2584]: [+]process-running ok
Jul 24 14:26:56.173491 ip-10-0-135-93 kubenswrapper[2584]: healthz check failed
Jul 24 14:26:56.173956 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.173513    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-df9867497-4w8nq" podUID="6eeb8c69-5d01-44e5-9fa3-77a9687c007b" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:26:56.259611 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.259575    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h6j7x"]
Jul 24 14:26:56.445892 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.445807    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.451254 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.451229    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 14:26:56.452370 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.452347    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gmr2w\""
Jul 24 14:26:56.452489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.452368    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 14:26:56.452489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.452412    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 14:26:56.452489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.452347    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 14:26:56.518996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.518959    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-sys\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.518996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.518995    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hcfh\" (UniqueName: \"kubernetes.io/projected/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-kube-api-access-4hcfh\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519027    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-tls\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519056    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-textfile\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519087    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-wtmp\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519194    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-root\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519224 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519222    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.519449 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.519261    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-metrics-client-ca\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620284 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620251    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-textfile\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620290    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620312    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-wtmp\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620348    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-root\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620373    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-metrics-client-ca\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-sys\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620458 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620440    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hcfh\" (UniqueName: \"kubernetes.io/projected/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-kube-api-access-4hcfh\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620458    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-root\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-tls\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620542    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-sys\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620577    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-wtmp\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.620800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.620670    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-textfile\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.621089 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.621066    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.621162 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.621126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-metrics-client-ca\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.622771 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.622748    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.622885 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.622803    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-node-exporter-tls\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.631813 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.631781    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hcfh\" (UniqueName: \"kubernetes.io/projected/0e97c5be-4c6b-4e1f-9eff-d517f9f63dee-kube-api-access-4hcfh\") pod \"node-exporter-h6j7x\" (UID: \"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee\") " pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.756095 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:56.756025    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6j7x"
Jul 24 14:26:56.827006 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:26:56.826979    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e97c5be_4c6b_4e1f_9eff_d517f9f63dee.slice/crio-947184b42b28139ba800fa56b3c12b1cef8e4961e442a4bf1a490168925231ca WatchSource:0}: Error finding container 947184b42b28139ba800fa56b3c12b1cef8e4961e442a4bf1a490168925231ca: Status 404 returned error can't find the container with id 947184b42b28139ba800fa56b3c12b1cef8e4961e442a4bf1a490168925231ca
Jul 24 14:26:57.173638 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:57.173611    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:57.174950 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:57.174930    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-df9867497-4w8nq"
Jul 24 14:26:57.517406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:57.517376    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6j7x" event={"ID":"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee","Type":"ContainerStarted","Data":"947184b42b28139ba800fa56b3c12b1cef8e4961e442a4bf1a490168925231ca"}
Jul 24 14:26:58.522186 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:58.522156    2584 generic.go:356] "Generic (PLEG): container finished" podID="0e97c5be-4c6b-4e1f-9eff-d517f9f63dee" containerID="705dcd4115f6aa8438b71d15b31f0e7b87718c325fcbd58e1f92b8e537e58770" exitCode=0
Jul 24 14:26:58.522659 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:58.522205    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6j7x" event={"ID":"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee","Type":"ContainerDied","Data":"705dcd4115f6aa8438b71d15b31f0e7b87718c325fcbd58e1f92b8e537e58770"}
Jul 24 14:26:59.527814 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:59.527771    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6j7x" event={"ID":"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee","Type":"ContainerStarted","Data":"2a1f3acd50189c54696fe3ea3a8991645cfb3ba822fa505ce86c324211e3cdff"}
Jul 24 14:26:59.528277 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:26:59.527877    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6j7x" event={"ID":"0e97c5be-4c6b-4e1f-9eff-d517f9f63dee","Type":"ContainerStarted","Data":"293e82541eded65e33a63f6da521e728b6335dc3995b8f0be3178e7da6204dac"}
Jul 24 14:27:06.759073 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:06.759022    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h6j7x" podStartSLOduration=9.772493973 podStartE2EDuration="10.759001588s" totalImagesPullingTime="922.731153ms" totalInitContainerRuntime="63.776462ms" isStatefulPod=true podCreationTimestamp="2026-07-24 14:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:26:59.577176883 +0000 UTC m=+276.361877384" watchObservedRunningTime="2026-07-24 14:27:06.759001588 +0000 UTC m=+283.543702054"
Jul 24 14:27:06.759999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:06.759981    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6754854769-k8g7q"]
Jul 24 14:27:09.565298 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:09.565266    2584 generic.go:356] "Generic (PLEG): container finished" podID="07b242ff-baae-4556-9d53-c483938af05e" containerID="0e21f85d48f4c8ac3cd0776227ad63038dfa9eeb4ebbcf6127b7a5e5fd9e19d7" exitCode=0
Jul 24 14:27:09.565666 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:09.565326    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9" event={"ID":"07b242ff-baae-4556-9d53-c483938af05e","Type":"ContainerDied","Data":"0e21f85d48f4c8ac3cd0776227ad63038dfa9eeb4ebbcf6127b7a5e5fd9e19d7"}
Jul 24 14:27:09.565666 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:09.565592    2584 scope.go:117] "RemoveContainer" containerID="0e21f85d48f4c8ac3cd0776227ad63038dfa9eeb4ebbcf6127b7a5e5fd9e19d7"
Jul 24 14:27:10.571486 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:10.571452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-st2k9" event={"ID":"07b242ff-baae-4556-9d53-c483938af05e","Type":"ContainerStarted","Data":"9dfdce78674a5b16a109bce92e683af654a1cd3d705ec9881bf35d1182d98be8"}
Jul 24 14:27:15.449438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:15.449401    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" podUID="9c2c4b78-ceef-45aa-aab3-bac4403e834d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:27:15.591350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:15.591320    2584 generic.go:356] "Generic (PLEG): container finished" podID="5d4ee50f-92ce-43fe-92f2-69391d789624" containerID="9d8b13fb29bb7cb5cfde3c509d3434c60b09067a38c8cd0766a54f7a72176fab" exitCode=0
Jul 24 14:27:15.591507 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:15.591398    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c" event={"ID":"5d4ee50f-92ce-43fe-92f2-69391d789624","Type":"ContainerDied","Data":"9d8b13fb29bb7cb5cfde3c509d3434c60b09067a38c8cd0766a54f7a72176fab"}
Jul 24 14:27:15.591751 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:15.591735    2584 scope.go:117] "RemoveContainer" containerID="9d8b13fb29bb7cb5cfde3c509d3434c60b09067a38c8cd0766a54f7a72176fab"
Jul 24 14:27:16.598009 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:16.597977    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c" event={"ID":"5d4ee50f-92ce-43fe-92f2-69391d789624","Type":"ContainerStarted","Data":"77ce636e0169ea2affdc0215509d7bcda791b557a5cf038933b1c2f754c9f471"}
Jul 24 14:27:23.513851 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:23.513808    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:27:23.516790 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:23.516765    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:27:23.526237 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:23.526211    2584 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 14:27:25.450663 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:25.450621    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" podUID="9c2c4b78-ceef-45aa-aab3-bac4403e834d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:27:31.778328 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:31.778289    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6754854769-k8g7q" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry" containerID="cri-o://20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c" gracePeriod=30
Jul 24 14:27:32.029034 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.028980    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:27:32.102143 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102118    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-installation-pull-secrets\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102143 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102147    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-certificates\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102192    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102217    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-image-registry-private-configuration\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102270    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-trusted-ca\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102286    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4f73506-ad36-43a8-8685-b5daf2987c1f-ca-trust-extracted\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102317    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-bound-sa-token\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102340    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxc5w\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-kube-api-access-gxc5w\") pod \"c4f73506-ad36-43a8-8685-b5daf2987c1f\" (UID: \"c4f73506-ad36-43a8-8685-b5daf2987c1f\") "
Jul 24 14:27:32.102728 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.102698    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:27:32.103212 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.103181    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:27:32.104893 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.104862    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-kube-api-access-gxc5w" (OuterVolumeSpecName: "kube-api-access-gxc5w") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "kube-api-access-gxc5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:27:32.104993 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.104962    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:27:32.105069 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.105026    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:27:32.105069 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.105039    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 14:27:32.105141 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.105110    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:27:32.111029 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.111002    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4f73506-ad36-43a8-8685-b5daf2987c1f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c4f73506-ad36-43a8-8685-b5daf2987c1f" (UID: "c4f73506-ad36-43a8-8685-b5daf2987c1f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:27:32.203563 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203543    2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-trusted-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203563 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203564    2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c4f73506-ad36-43a8-8685-b5daf2987c1f-ca-trust-extracted\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203574    2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-bound-sa-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203583    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxc5w\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-kube-api-access-gxc5w\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203591    2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-installation-pull-secrets\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203599    2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-certificates\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203607    2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c4f73506-ad36-43a8-8685-b5daf2987c1f-registry-tls\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.203683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.203617    2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c4f73506-ad36-43a8-8685-b5daf2987c1f-image-registry-private-configuration\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:27:32.656267 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.656219    2584 generic.go:356] "Generic (PLEG): container finished" podID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerID="20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c" exitCode=0
Jul 24 14:27:32.656267 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.656263    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6754854769-k8g7q"
Jul 24 14:27:32.656506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.656295    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6754854769-k8g7q" event={"ID":"c4f73506-ad36-43a8-8685-b5daf2987c1f","Type":"ContainerDied","Data":"20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c"}
Jul 24 14:27:32.656506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.656344    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6754854769-k8g7q" event={"ID":"c4f73506-ad36-43a8-8685-b5daf2987c1f","Type":"ContainerDied","Data":"813ac890aeed92cba2ab978edc483ad319c23339ce6ba72b761e58103a23677d"}
Jul 24 14:27:32.656506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.656370    2584 scope.go:117] "RemoveContainer" containerID="20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c"
Jul 24 14:27:32.664771 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.664756    2584 scope.go:117] "RemoveContainer" containerID="20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c"
Jul 24 14:27:32.665070 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:27:32.665050    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c\": container with ID starting with 20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c not found: ID does not exist" containerID="20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c"
Jul 24 14:27:32.665134 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.665077    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c"} err="failed to get container status \"20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c\": rpc error: code = NotFound desc = could not find container \"20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c\": container with ID starting with 20f7a795542a83c19ba1ff7e14686aba7d5ccaeaf752f9991d517c7612110e0c not found: ID does not exist"
Jul 24 14:27:32.682677 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.682657    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6754854769-k8g7q"]
Jul 24 14:27:32.688026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:32.688006    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6754854769-k8g7q"]
Jul 24 14:27:33.669356 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:33.669326    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" path="/var/lib/kubelet/pods/c4f73506-ad36-43a8-8685-b5daf2987c1f/volumes"
Jul 24 14:27:34.664437 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:34.664391    2584 generic.go:356] "Generic (PLEG): container finished" podID="a09f2c2a-eb19-440c-8cf2-24d6f4328003" containerID="25248090ac620724030b90462e9cfe083be3f6a324eb31f444470b993a632cdf" exitCode=0
Jul 24 14:27:34.664605 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:34.664487    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-j9lx2" event={"ID":"a09f2c2a-eb19-440c-8cf2-24d6f4328003","Type":"ContainerDied","Data":"25248090ac620724030b90462e9cfe083be3f6a324eb31f444470b993a632cdf"}
Jul 24 14:27:34.664814 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:34.664796    2584 scope.go:117] "RemoveContainer" containerID="25248090ac620724030b90462e9cfe083be3f6a324eb31f444470b993a632cdf"
Jul 24 14:27:35.448963 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.448920    2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" podUID="9c2c4b78-ceef-45aa-aab3-bac4403e834d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 24 14:27:35.449345 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.449004    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788"
Jul 24 14:27:35.449605 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.449586    2584 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a0af074f163b62e6fec7aeaced56461ba1b5b41c1ddf9496db3ef8f56797c7b4"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 24 14:27:35.449642 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.449629    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" podUID="9c2c4b78-ceef-45aa-aab3-bac4403e834d" containerName="service-proxy" containerID="cri-o://a0af074f163b62e6fec7aeaced56461ba1b5b41c1ddf9496db3ef8f56797c7b4" gracePeriod=30
Jul 24 14:27:35.670980 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.670928    2584 generic.go:356] "Generic (PLEG): container finished" podID="9c2c4b78-ceef-45aa-aab3-bac4403e834d" containerID="a0af074f163b62e6fec7aeaced56461ba1b5b41c1ddf9496db3ef8f56797c7b4" exitCode=2
Jul 24 14:27:35.671132 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.671008    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" event={"ID":"9c2c4b78-ceef-45aa-aab3-bac4403e834d","Type":"ContainerDied","Data":"a0af074f163b62e6fec7aeaced56461ba1b5b41c1ddf9496db3ef8f56797c7b4"}
Jul 24 14:27:35.671132 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.671055    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b594c5f58-w6788" event={"ID":"9c2c4b78-ceef-45aa-aab3-bac4403e834d","Type":"ContainerStarted","Data":"6f340ced527d4c960cd724ec8d688b08fe0bee63c16484ce96f8ad7bab1c1303"}
Jul 24 14:27:35.673431 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.673402    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-j9lx2" event={"ID":"a09f2c2a-eb19-440c-8cf2-24d6f4328003","Type":"ContainerStarted","Data":"523c9ffd236af86b84c30b8884b8c4cc276b4d086fd1338466258c1da2d69b1d"}
Jul 24 14:27:35.718991 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.718919    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-df9867497-4w8nq_6eeb8c69-5d01-44e5-9fa3-77a9687c007b/router/0.log"
Jul 24 14:27:35.739931 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:35.739906    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbkpq_ea5c9656-f807-497c-8f41-715f74b4385f/serve-healthcheck-canary/0.log"
Jul 24 14:27:58.758057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:58.758024    2584 generic.go:356] "Generic (PLEG): container finished" podID="5b52fc62-1a70-435a-9ccc-11d17011c038" containerID="349ef96a8eb3c6d803c4854071cd868a4dca38473fcc5ee290e545df0604d8e5" exitCode=1
Jul 24 14:27:58.758471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:58.758096    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" event={"ID":"5b52fc62-1a70-435a-9ccc-11d17011c038","Type":"ContainerDied","Data":"349ef96a8eb3c6d803c4854071cd868a4dca38473fcc5ee290e545df0604d8e5"}
Jul 24 14:27:58.758471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:58.758420    2584 scope.go:117] "RemoveContainer" containerID="349ef96a8eb3c6d803c4854071cd868a4dca38473fcc5ee290e545df0604d8e5"
Jul 24 14:27:58.759569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:58.759547    2584 generic.go:356] "Generic (PLEG): container finished" podID="ed673a39-32ca-4685-b89c-f5de447ea0c8" containerID="e53d7ea770f4ffa15588b2434a8e9afbd0db508ed50e185ddd73195184a07ddd" exitCode=255
Jul 24 14:27:58.759663 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:58.759625    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" event={"ID":"ed673a39-32ca-4685-b89c-f5de447ea0c8","Type":"ContainerDied","Data":"e53d7ea770f4ffa15588b2434a8e9afbd0db508ed50e185ddd73195184a07ddd"}
Jul 24 14:27:58.759901 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:58.759888    2584 scope.go:117] "RemoveContainer" containerID="e53d7ea770f4ffa15588b2434a8e9afbd0db508ed50e185ddd73195184a07ddd"
Jul 24 14:27:59.228412 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:59.228380    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:27:59.765175 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:59.765144    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" event={"ID":"5b52fc62-1a70-435a-9ccc-11d17011c038","Type":"ContainerStarted","Data":"d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5"}
Jul 24 14:27:59.765637 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:59.765328    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:27:59.765961 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:59.765936    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:27:59.767050 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:27:59.767027    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" event={"ID":"ed673a39-32ca-4685-b89c-f5de447ea0c8","Type":"ContainerStarted","Data":"478e7eeaf4e575107439e8271def467cb66fed7da7c5f1f737866fb9a28b6ce8"}
Jul 24 14:29:59.221726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.221640    2584 generic.go:356] "Generic (PLEG): container finished" podID="5b52fc62-1a70-435a-9ccc-11d17011c038" containerID="d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5" exitCode=1
Jul 24 14:29:59.222251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.221722    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" event={"ID":"5b52fc62-1a70-435a-9ccc-11d17011c038","Type":"ContainerDied","Data":"d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5"}
Jul 24 14:29:59.222251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.221792    2584 scope.go:117] "RemoveContainer" containerID="349ef96a8eb3c6d803c4854071cd868a4dca38473fcc5ee290e545df0604d8e5"
Jul 24 14:29:59.222251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.222192    2584 scope.go:117] "RemoveContainer" containerID="d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5"
Jul 24 14:29:59.222411 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:29:59.222392    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-788fb66f64-bb9wm_open-cluster-management-agent-addon(5b52fc62-1a70-435a-9ccc-11d17011c038)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" podUID="5b52fc62-1a70-435a-9ccc-11d17011c038"
Jul 24 14:29:59.223498 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.223472    2584 generic.go:356] "Generic (PLEG): container finished" podID="ed673a39-32ca-4685-b89c-f5de447ea0c8" containerID="478e7eeaf4e575107439e8271def467cb66fed7da7c5f1f737866fb9a28b6ce8" exitCode=255
Jul 24 14:29:59.223600 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.223505    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" event={"ID":"ed673a39-32ca-4685-b89c-f5de447ea0c8","Type":"ContainerDied","Data":"478e7eeaf4e575107439e8271def467cb66fed7da7c5f1f737866fb9a28b6ce8"}
Jul 24 14:29:59.223758 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.223747    2584 scope.go:117] "RemoveContainer" containerID="478e7eeaf4e575107439e8271def467cb66fed7da7c5f1f737866fb9a28b6ce8"
Jul 24 14:29:59.223926 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:29:59.223910    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-776c947c66-5gpxn_open-cluster-management-agent-addon(ed673a39-32ca-4685-b89c-f5de447ea0c8)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" podUID="ed673a39-32ca-4685-b89c-f5de447ea0c8"
Jul 24 14:29:59.234652 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.234635    2584 scope.go:117] "RemoveContainer" containerID="e53d7ea770f4ffa15588b2434a8e9afbd0db508ed50e185ddd73195184a07ddd"
Jul 24 14:29:59.765436 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:29:59.765415    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:30:00.233637 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:00.233610    2584 scope.go:117] "RemoveContainer" containerID="d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5"
Jul 24 14:30:00.234081 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:00.233793    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-788fb66f64-bb9wm_open-cluster-management-agent-addon(5b52fc62-1a70-435a-9ccc-11d17011c038)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" podUID="5b52fc62-1a70-435a-9ccc-11d17011c038"
Jul 24 14:30:05.415275 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:05.415239    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn"
Jul 24 14:30:05.415734 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:05.415627    2584 scope.go:117] "RemoveContainer" containerID="478e7eeaf4e575107439e8271def467cb66fed7da7c5f1f737866fb9a28b6ce8"
Jul 24 14:30:05.415821 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:05.415802    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-776c947c66-5gpxn_open-cluster-management-agent-addon(ed673a39-32ca-4685-b89c-f5de447ea0c8)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" podUID="ed673a39-32ca-4685-b89c-f5de447ea0c8"
Jul 24 14:30:05.431597 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:05.431576    2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:30:05.431910 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:05.431897    2584 scope.go:117] "RemoveContainer" containerID="d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5"
Jul 24 14:30:05.432073 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:05.432056    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-788fb66f64-bb9wm_open-cluster-management-agent-addon(5b52fc62-1a70-435a-9ccc-11d17011c038)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" podUID="5b52fc62-1a70-435a-9ccc-11d17011c038"
Jul 24 14:30:16.665187 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:16.665155    2584 scope.go:117] "RemoveContainer" containerID="d6a30a31ca912731f17b8da514ad8f52fa5e473719314fe762998ed489356cc5"
Jul 24 14:30:17.298240 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:17.298210    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm" event={"ID":"5b52fc62-1a70-435a-9ccc-11d17011c038","Type":"ContainerStarted","Data":"487994d3ab1dac9a63aaf6e1c95a2b5e9e1f0475c412f45595295c0fa3bf7427"}
Jul 24 14:30:17.298509 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:17.298489    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:30:17.300321 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:17.300302    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-788fb66f64-bb9wm"
Jul 24 14:30:19.665251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:19.665220    2584 scope.go:117] "RemoveContainer" containerID="478e7eeaf4e575107439e8271def467cb66fed7da7c5f1f737866fb9a28b6ce8"
Jul 24 14:30:20.310410 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:20.310377    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-776c947c66-5gpxn" event={"ID":"ed673a39-32ca-4685-b89c-f5de447ea0c8","Type":"ContainerStarted","Data":"5e6accd856ba01e1e935195d16a96c09f4d1749925be0db5c0e76d94dbc61695"}
Jul 24 14:30:27.311896 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.311858    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"]
Jul 24 14:30:27.312460 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.312304    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry"
Jul 24 14:30:27.312460 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.312322    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry"
Jul 24 14:30:27.312460 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.312398    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4f73506-ad36-43a8-8685-b5daf2987c1f" containerName="registry"
Jul 24 14:30:27.315501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.315481    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.318660 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.318636    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 14:30:27.318660 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.318650    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k4dnd\""
Jul 24 14:30:27.318811 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.318751    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 14:30:27.318964 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.318951    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 14:30:27.319025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.318991    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 14:30:27.329384 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.329363    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"]
Jul 24 14:30:27.395778 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.395751    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f858af5-17df-41c1-ad70-20ade164f31c-tmp\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.395961 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.395822    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f858af5-17df-41c1-ad70-20ade164f31c-webhook-cert\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.395961 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.395885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f858af5-17df-41c1-ad70-20ade164f31c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.395961 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.395903    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f858af5-17df-41c1-ad70-20ade164f31c-manifests\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.395961 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.395936    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsx99\" (UniqueName: \"kubernetes.io/projected/6f858af5-17df-41c1-ad70-20ade164f31c-kube-api-access-lsx99\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.497357 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.497320    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f858af5-17df-41c1-ad70-20ade164f31c-tmp\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.497819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.497381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f858af5-17df-41c1-ad70-20ade164f31c-webhook-cert\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.497819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.497400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f858af5-17df-41c1-ad70-20ade164f31c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.497819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.497519    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f858af5-17df-41c1-ad70-20ade164f31c-manifests\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.497819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.497570    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx99\" (UniqueName: \"kubernetes.io/projected/6f858af5-17df-41c1-ad70-20ade164f31c-kube-api-access-lsx99\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.498077 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.497947    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6f858af5-17df-41c1-ad70-20ade164f31c-manifests\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.499725 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.499702    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f858af5-17df-41c1-ad70-20ade164f31c-tmp\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.499863 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.499736    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6f858af5-17df-41c1-ad70-20ade164f31c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.499932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.499906    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6f858af5-17df-41c1-ad70-20ade164f31c-webhook-cert\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.508930 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.508905    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsx99\" (UniqueName: \"kubernetes.io/projected/6f858af5-17df-41c1-ad70-20ade164f31c-kube-api-access-lsx99\") pod \"opendatahub-operator-controller-manager-64bbcbb69d-v9hj7\" (UID: \"6f858af5-17df-41c1-ad70-20ade164f31c\") " pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.625369 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.625267    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:27.778025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.777993    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"]
Jul 24 14:30:27.781679 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:30:27.781652    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f858af5_17df_41c1_ad70_20ade164f31c.slice/crio-aa65cd7c853a018fa6e0d3c4a66437e299e86dfb947b4c9a1c43055974b19a49 WatchSource:0}: Error finding container aa65cd7c853a018fa6e0d3c4a66437e299e86dfb947b4c9a1c43055974b19a49: Status 404 returned error can't find the container with id aa65cd7c853a018fa6e0d3c4a66437e299e86dfb947b4c9a1c43055974b19a49
Jul 24 14:30:27.783424 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:27.783409    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:30:28.350941 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:28.350884    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7" event={"ID":"6f858af5-17df-41c1-ad70-20ade164f31c","Type":"ContainerStarted","Data":"aa65cd7c853a018fa6e0d3c4a66437e299e86dfb947b4c9a1c43055974b19a49"}
Jul 24 14:30:31.364800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:31.364767    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7" event={"ID":"6f858af5-17df-41c1-ad70-20ade164f31c","Type":"ContainerStarted","Data":"53ae09872882e8e8db979cd8c881b77cf4aff2fdd5ed118d2e9be8141282ecb5"}
Jul 24 14:30:32.370789 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:32.370756    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f858af5-17df-41c1-ad70-20ade164f31c" containerID="53ae09872882e8e8db979cd8c881b77cf4aff2fdd5ed118d2e9be8141282ecb5" exitCode=0
Jul 24 14:30:32.371192 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:32.370856    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7" event={"ID":"6f858af5-17df-41c1-ad70-20ade164f31c","Type":"ContainerDied","Data":"53ae09872882e8e8db979cd8c881b77cf4aff2fdd5ed118d2e9be8141282ecb5"}
Jul 24 14:30:33.376973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:33.376940    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7" event={"ID":"6f858af5-17df-41c1-ad70-20ade164f31c","Type":"ContainerStarted","Data":"a847ac7bbff1a4194932654fc120865e22eebeb4b1482f3af131d52fb07a7998"}
Jul 24 14:30:33.377374 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:33.377065    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:33.410426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:33.410363    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7" podStartSLOduration=2.16180461 podStartE2EDuration="6.410340793s" totalImagesPullingTime="3.218458184s" totalInitContainerRuntime="1.030077999s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:30:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:30:33.409106298 +0000 UTC m=+490.193806762" watchObservedRunningTime="2026-07-24 14:30:33.410340793 +0000 UTC m=+490.195041258"
Jul 24 14:30:44.388925 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:44.388896    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64bbcbb69d-v9hj7"
Jul 24 14:30:47.602655 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.602615    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fvgbg"]
Jul 24 14:30:47.606352 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.606325    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.612137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.612119    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 24 14:30:47.612276 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.612255    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8cl59\""
Jul 24 14:30:47.624473 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.624446    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fvgbg"]
Jul 24 14:30:47.652932 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.652902    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2jcm\" (UniqueName: \"kubernetes.io/projected/410dbd66-021d-43a4-b47b-471cee57ab7d-kube-api-access-j2jcm\") pod \"odh-model-controller-7998c67dc4-fvgbg\" (UID: \"410dbd66-021d-43a4-b47b-471cee57ab7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.653087 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.652950    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/410dbd66-021d-43a4-b47b-471cee57ab7d-cert\") pod \"odh-model-controller-7998c67dc4-fvgbg\" (UID: \"410dbd66-021d-43a4-b47b-471cee57ab7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.753585 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.753547    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2jcm\" (UniqueName: \"kubernetes.io/projected/410dbd66-021d-43a4-b47b-471cee57ab7d-kube-api-access-j2jcm\") pod \"odh-model-controller-7998c67dc4-fvgbg\" (UID: \"410dbd66-021d-43a4-b47b-471cee57ab7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.753585 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.753590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/410dbd66-021d-43a4-b47b-471cee57ab7d-cert\") pod \"odh-model-controller-7998c67dc4-fvgbg\" (UID: \"410dbd66-021d-43a4-b47b-471cee57ab7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.755962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.755938    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/410dbd66-021d-43a4-b47b-471cee57ab7d-cert\") pod \"odh-model-controller-7998c67dc4-fvgbg\" (UID: \"410dbd66-021d-43a4-b47b-471cee57ab7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.764463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.764438    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2jcm\" (UniqueName: \"kubernetes.io/projected/410dbd66-021d-43a4-b47b-471cee57ab7d-kube-api-access-j2jcm\") pod \"odh-model-controller-7998c67dc4-fvgbg\" (UID: \"410dbd66-021d-43a4-b47b-471cee57ab7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:47.919027 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:47.918949    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:48.063092 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.063067    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fvgbg"]
Jul 24 14:30:48.065063 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:30:48.065034    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410dbd66_021d_43a4_b47b_471cee57ab7d.slice/crio-aa2d03e2a302d7254c5a8d46c7590ce4f4b117e4d2d6b0ca46bca6729e30f22f WatchSource:0}: Error finding container aa2d03e2a302d7254c5a8d46c7590ce4f4b117e4d2d6b0ca46bca6729e30f22f: Status 404 returned error can't find the container with id aa2d03e2a302d7254c5a8d46c7590ce4f4b117e4d2d6b0ca46bca6729e30f22f
Jul 24 14:30:48.368691 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.368662    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4mrl2"]
Jul 24 14:30:48.373167 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.373150    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:48.375967 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.375952    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 14:30:48.386411 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.386391    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4mrl2"]
Jul 24 14:30:48.448992 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.448956    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg" event={"ID":"410dbd66-021d-43a4-b47b-471cee57ab7d","Type":"ContainerStarted","Data":"aa2d03e2a302d7254c5a8d46c7590ce4f4b117e4d2d6b0ca46bca6729e30f22f"}
Jul 24 14:30:48.459704 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.459678    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b667428-c4e6-4e34-8682-7f48060e6526-tls-certs\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:48.459863 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.459715    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s54sg\" (UniqueName: \"kubernetes.io/projected/6b667428-c4e6-4e34-8682-7f48060e6526-kube-api-access-s54sg\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:48.560916 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.560881    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b667428-c4e6-4e34-8682-7f48060e6526-tls-certs\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:48.560916 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.560921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s54sg\" (UniqueName: \"kubernetes.io/projected/6b667428-c4e6-4e34-8682-7f48060e6526-kube-api-access-s54sg\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:48.561130 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:48.561049    2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 24 14:30:48.561130 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:48.561129    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b667428-c4e6-4e34-8682-7f48060e6526-tls-certs podName:6b667428-c4e6-4e34-8682-7f48060e6526 nodeName:}" failed. No retries permitted until 2026-07-24 14:30:49.06110528 +0000 UTC m=+505.845805734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6b667428-c4e6-4e34-8682-7f48060e6526-tls-certs") pod "model-serving-api-54d89bd5fc-4mrl2" (UID: "6b667428-c4e6-4e34-8682-7f48060e6526") : secret "model-serving-api-tls" not found
Jul 24 14:30:48.574996 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:48.574969    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s54sg\" (UniqueName: \"kubernetes.io/projected/6b667428-c4e6-4e34-8682-7f48060e6526-kube-api-access-s54sg\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:49.065737 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:49.065701    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b667428-c4e6-4e34-8682-7f48060e6526-tls-certs\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:49.069081 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:49.069034    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b667428-c4e6-4e34-8682-7f48060e6526-tls-certs\") pod \"model-serving-api-54d89bd5fc-4mrl2\" (UID: \"6b667428-c4e6-4e34-8682-7f48060e6526\") " pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:49.286495 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:49.286447    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:49.445784 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:49.445759    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4mrl2"]
Jul 24 14:30:50.572142 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:30:50.572111    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b667428_c4e6_4e34_8682_7f48060e6526.slice/crio-b20625a8e25644465df9548749416c52eb7fb1255c54e746aa3e2633903cf49c WatchSource:0}: Error finding container b20625a8e25644465df9548749416c52eb7fb1255c54e746aa3e2633903cf49c: Status 404 returned error can't find the container with id b20625a8e25644465df9548749416c52eb7fb1255c54e746aa3e2633903cf49c
Jul 24 14:30:51.464936 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:51.464906    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2" event={"ID":"6b667428-c4e6-4e34-8682-7f48060e6526","Type":"ContainerStarted","Data":"b20625a8e25644465df9548749416c52eb7fb1255c54e746aa3e2633903cf49c"}
Jul 24 14:30:51.466317 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:51.466289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg" event={"ID":"410dbd66-021d-43a4-b47b-471cee57ab7d","Type":"ContainerStarted","Data":"9087d3ea44832bd5dd9afb535374de96710055fae95d050346c7a5df9ef8d997"}
Jul 24 14:30:51.466450 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:51.466396    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:30:52.474342 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:52.474256    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2" event={"ID":"6b667428-c4e6-4e34-8682-7f48060e6526","Type":"ContainerStarted","Data":"c38270a3a3536c06bd7094608722057e229a882d92b1d81c2e9a912ff0a55a18"}
Jul 24 14:30:52.474788 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:52.474343    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:30:52.496472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:52.496422    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2" podStartSLOduration=2.947015909 podStartE2EDuration="4.496405754s" totalImagesPullingTime="1.549389845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:30:52.49596427 +0000 UTC m=+509.280664734" watchObservedRunningTime="2026-07-24 14:30:52.496405754 +0000 UTC m=+509.281106219"
Jul 24 14:30:52.497501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:52.497458    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg" podStartSLOduration=2.933668854 podStartE2EDuration="5.497447605s" totalImagesPullingTime="2.563778751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:30:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:30:51.48564114 +0000 UTC m=+508.270341604" watchObservedRunningTime="2026-07-24 14:30:52.497447605 +0000 UTC m=+509.282148074"
Jul 24 14:30:58.076003 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.075970    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"]
Jul 24 14:30:58.079344 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.079329    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.081658 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.081632    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 14:30:58.081777 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.081635    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 14:30:58.082606 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.082585    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7kwwm\""
Jul 24 14:30:58.088961 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.088941    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"]
Jul 24 14:30:58.144318 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.144290    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5483efb5-b239-4f23-92ad-1d3253577f4e-tmp\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.144465 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.144335    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5483efb5-b239-4f23-92ad-1d3253577f4e-tls-certs\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.144465 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.144409    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nmrq\" (UniqueName: \"kubernetes.io/projected/5483efb5-b239-4f23-92ad-1d3253577f4e-kube-api-access-4nmrq\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.245389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.245359    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5483efb5-b239-4f23-92ad-1d3253577f4e-tls-certs\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.245576 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.245411    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nmrq\" (UniqueName: \"kubernetes.io/projected/5483efb5-b239-4f23-92ad-1d3253577f4e-kube-api-access-4nmrq\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.245576 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.245450    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5483efb5-b239-4f23-92ad-1d3253577f4e-tmp\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.247755 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.247731    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5483efb5-b239-4f23-92ad-1d3253577f4e-tmp\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.248034 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.248011    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5483efb5-b239-4f23-92ad-1d3253577f4e-tls-certs\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.258704 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.258680    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nmrq\" (UniqueName: \"kubernetes.io/projected/5483efb5-b239-4f23-92ad-1d3253577f4e-kube-api-access-4nmrq\") pod \"kube-auth-proxy-68b7bd974d-7rzwd\" (UID: \"5483efb5-b239-4f23-92ad-1d3253577f4e\") " pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.389572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.389495    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"
Jul 24 14:30:58.533391 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:58.533362    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd"]
Jul 24 14:30:58.536503 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:30:58.536475    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5483efb5_b239_4f23_92ad_1d3253577f4e.slice/crio-c3f6fccef897081cdce66776b179a8fc57d91959fb8348cb559d6c65f684bd4e WatchSource:0}: Error finding container c3f6fccef897081cdce66776b179a8fc57d91959fb8348cb559d6c65f684bd4e: Status 404 returned error can't find the container with id c3f6fccef897081cdce66776b179a8fc57d91959fb8348cb559d6c65f684bd4e
Jul 24 14:30:59.506692 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.506161    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd" event={"ID":"5483efb5-b239-4f23-92ad-1d3253577f4e","Type":"ContainerStarted","Data":"c3f6fccef897081cdce66776b179a8fc57d91959fb8348cb559d6c65f684bd4e"}
Jul 24 14:30:59.579166 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.579133    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"]
Jul 24 14:30:59.583735 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.583714    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.585791 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.584409    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"]
Jul 24 14:30:59.586806 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.586778    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-s4jnc\""
Jul 24 14:30:59.587382 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.587364    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 14:30:59.589666 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.589586    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:30:59.598046 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.597634    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5nh92\""
Jul 24 14:30:59.598046 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.597898    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 14:30:59.613742 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.613719    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"]
Jul 24 14:30:59.615439 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.615379    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"]
Jul 24 14:30:59.657238 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.657209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rw5q\" (UniqueName: \"kubernetes.io/projected/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-kube-api-access-4rw5q\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:30:59.657360 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.657260    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj7f5\" (UniqueName: \"kubernetes.io/projected/6b265693-dfd3-4099-a377-4bfaf207ad57-kube-api-access-gj7f5\") pod \"kserve-controller-manager-d6f4b65b8-74vtf\" (UID: \"6b265693-dfd3-4099-a377-4bfaf207ad57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.657360 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.657303    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:30:59.657463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.657413    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b265693-dfd3-4099-a377-4bfaf207ad57-cert\") pod \"kserve-controller-manager-d6f4b65b8-74vtf\" (UID: \"6b265693-dfd3-4099-a377-4bfaf207ad57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.758246 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.758155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:30:59.758405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.758271    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b265693-dfd3-4099-a377-4bfaf207ad57-cert\") pod \"kserve-controller-manager-d6f4b65b8-74vtf\" (UID: \"6b265693-dfd3-4099-a377-4bfaf207ad57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.758405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.758319    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rw5q\" (UniqueName: \"kubernetes.io/projected/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-kube-api-access-4rw5q\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:30:59.758405 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:59.758324    2584 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 14:30:59.758405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.758357    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj7f5\" (UniqueName: \"kubernetes.io/projected/6b265693-dfd3-4099-a377-4bfaf207ad57-kube-api-access-gj7f5\") pod \"kserve-controller-manager-d6f4b65b8-74vtf\" (UID: \"6b265693-dfd3-4099-a377-4bfaf207ad57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.758405 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:30:59.758400    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert podName:d6dbdcdc-fd02-4e07-b4a8-b05aca23a968 nodeName:}" failed. No retries permitted until 2026-07-24 14:31:00.258376628 +0000 UTC m=+517.043077077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert") pod "llmisvc-controller-manager-777c6bd7d5-j98hz" (UID: "d6dbdcdc-fd02-4e07-b4a8-b05aca23a968") : secret "llmisvc-webhook-server-cert" not found
Jul 24 14:30:59.761254 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.761203    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b265693-dfd3-4099-a377-4bfaf207ad57-cert\") pod \"kserve-controller-manager-d6f4b65b8-74vtf\" (UID: \"6b265693-dfd3-4099-a377-4bfaf207ad57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.782970 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.782930    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj7f5\" (UniqueName: \"kubernetes.io/projected/6b265693-dfd3-4099-a377-4bfaf207ad57-kube-api-access-gj7f5\") pod \"kserve-controller-manager-d6f4b65b8-74vtf\" (UID: \"6b265693-dfd3-4099-a377-4bfaf207ad57\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:30:59.784274 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.784228    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rw5q\" (UniqueName: \"kubernetes.io/projected/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-kube-api-access-4rw5q\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:30:59.919095 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:30:59.919056    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:31:00.117148 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:00.117057    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"]
Jul 24 14:31:00.119695 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:31:00.119658    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b265693_dfd3_4099_a377_4bfaf207ad57.slice/crio-ed24e3f75a26018a6a0ecaa974e9430ee8932668b958c63a39ee6bfac3940499 WatchSource:0}: Error finding container ed24e3f75a26018a6a0ecaa974e9430ee8932668b958c63a39ee6bfac3940499: Status 404 returned error can't find the container with id ed24e3f75a26018a6a0ecaa974e9430ee8932668b958c63a39ee6bfac3940499
Jul 24 14:31:00.263272 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:00.263158    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:31:00.263427 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:31:00.263337    2584 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 14:31:00.263427 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:31:00.263417    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert podName:d6dbdcdc-fd02-4e07-b4a8-b05aca23a968 nodeName:}" failed. No retries permitted until 2026-07-24 14:31:01.263396252 +0000 UTC m=+518.048096697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert") pod "llmisvc-controller-manager-777c6bd7d5-j98hz" (UID: "d6dbdcdc-fd02-4e07-b4a8-b05aca23a968") : secret "llmisvc-webhook-server-cert" not found
Jul 24 14:31:00.511803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:00.511760    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf" event={"ID":"6b265693-dfd3-4099-a377-4bfaf207ad57","Type":"ContainerStarted","Data":"ed24e3f75a26018a6a0ecaa974e9430ee8932668b958c63a39ee6bfac3940499"}
Jul 24 14:31:01.273218 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:01.273179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:31:01.276214 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:01.276186    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6dbdcdc-fd02-4e07-b4a8-b05aca23a968-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j98hz\" (UID: \"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:31:01.425888 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:01.425864    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:31:01.519047 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:01.519008    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd" event={"ID":"5483efb5-b239-4f23-92ad-1d3253577f4e","Type":"ContainerStarted","Data":"aa4a7dc8e1729d631757f1849a3ff51564d7e9146c496c485f70835a09b648ea"}
Jul 24 14:31:01.591912 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:01.591804    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68b7bd974d-7rzwd" podStartSLOduration=0.772860889 podStartE2EDuration="3.591777386s" totalImagesPullingTime="2.818916497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:31:01.559128805 +0000 UTC m=+518.343829269" watchObservedRunningTime="2026-07-24 14:31:01.591777386 +0000 UTC m=+518.376477850"
Jul 24 14:31:01.592245 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:01.592229    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"]
Jul 24 14:31:01.595930 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:31:01.595898    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd6dbdcdc_fd02_4e07_b4a8_b05aca23a968.slice/crio-0cf7cbe1d7a43cde2c335eaa1c42bdb3c92f277870e475c941e4620e91cacdae WatchSource:0}: Error finding container 0cf7cbe1d7a43cde2c335eaa1c42bdb3c92f277870e475c941e4620e91cacdae: Status 404 returned error can't find the container with id 0cf7cbe1d7a43cde2c335eaa1c42bdb3c92f277870e475c941e4620e91cacdae
Jul 24 14:31:02.476138 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:02.476116    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fvgbg"
Jul 24 14:31:02.527819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:02.527781    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf" event={"ID":"6b265693-dfd3-4099-a377-4bfaf207ad57","Type":"ContainerStarted","Data":"4d813af9a4ab3fbb1f2dfdde1d4ceb88213a2b333422ada3554c46c1fc39221b"}
Jul 24 14:31:02.528206 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:02.527867    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:31:02.529055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:02.529033    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz" event={"ID":"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968","Type":"ContainerStarted","Data":"0cf7cbe1d7a43cde2c335eaa1c42bdb3c92f277870e475c941e4620e91cacdae"}
Jul 24 14:31:02.620047 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:02.619988    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf" podStartSLOduration=1.337559985 podStartE2EDuration="3.61996568s" totalImagesPullingTime="2.282405695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:31:02.619947245 +0000 UTC m=+519.404647709" watchObservedRunningTime="2026-07-24 14:31:02.61996568 +0000 UTC m=+519.404666148"
Jul 24 14:31:03.482943 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:03.482912    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4mrl2"
Jul 24 14:31:05.548397 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:05.548356    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz" event={"ID":"d6dbdcdc-fd02-4e07-b4a8-b05aca23a968","Type":"ContainerStarted","Data":"8ebbef61f80c876922dd58fed21ebb5bb12838e1f4d94d6736ab56e3734ba86f"}
Jul 24 14:31:05.548801 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:05.548424    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:31:05.576923 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:05.576876    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz" podStartSLOduration=3.475987866 podStartE2EDuration="6.576860914s" totalImagesPullingTime="3.100873048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:31:05.576532233 +0000 UTC m=+522.361232697" watchObservedRunningTime="2026-07-24 14:31:05.576860914 +0000 UTC m=+522.361561369"
Jul 24 14:31:06.194168 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.194137    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"]
Jul 24 14:31:06.197594 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.197574    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.200388 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.200362    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lk8bq\""
Jul 24 14:31:06.200481 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.200447    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 14:31:06.213368 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.213345    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"]
Jul 24 14:31:06.314351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314313    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314354    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642f2c92-f994-4277-8613-d7371432db23-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314374    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642f2c92-f994-4277-8613-d7371432db23-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314461    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314502    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314529    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314554    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642f2c92-f994-4277-8613-d7371432db23-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314734 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314609    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.314734 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.314627    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfl26\" (UniqueName: \"kubernetes.io/projected/642f2c92-f994-4277-8613-d7371432db23-kube-api-access-nfl26\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.415763 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.415736    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.415763 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.415768    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642f2c92-f994-4277-8613-d7371432db23-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416011 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.415795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642f2c92-f994-4277-8613-d7371432db23-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416011 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.415852    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416029    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416086    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416230 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416130    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642f2c92-f994-4277-8613-d7371432db23-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416230 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416185    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416230 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416219    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfl26\" (UniqueName: \"kubernetes.io/projected/642f2c92-f994-4277-8613-d7371432db23-kube-api-access-nfl26\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416377 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416267    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416377 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416335    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416476 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416411    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416609 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416590    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.416671 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.416616    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642f2c92-f994-4277-8613-d7371432db23-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.418063 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.418045    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642f2c92-f994-4277-8613-d7371432db23-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.418315 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.418297    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642f2c92-f994-4277-8613-d7371432db23-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.427309 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.427287    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfl26\" (UniqueName: \"kubernetes.io/projected/642f2c92-f994-4277-8613-d7371432db23-kube-api-access-nfl26\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.427917 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.427893    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642f2c92-f994-4277-8613-d7371432db23-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq\" (UID: \"642f2c92-f994-4277-8613-d7371432db23\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.510643 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.510563    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:06.664296 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:06.664273    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"]
Jul 24 14:31:06.666243 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:31:06.666212    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642f2c92_f994_4277_8613_d7371432db23.slice/crio-4c2c85a011c8ed76352f8bcfb92e79b0bf363529ebc0bfb050c2e15a2a3ae9a5 WatchSource:0}: Error finding container 4c2c85a011c8ed76352f8bcfb92e79b0bf363529ebc0bfb050c2e15a2a3ae9a5: Status 404 returned error can't find the container with id 4c2c85a011c8ed76352f8bcfb92e79b0bf363529ebc0bfb050c2e15a2a3ae9a5
Jul 24 14:31:07.559051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:07.559020    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq" event={"ID":"642f2c92-f994-4277-8613-d7371432db23","Type":"ContainerStarted","Data":"4c2c85a011c8ed76352f8bcfb92e79b0bf363529ebc0bfb050c2e15a2a3ae9a5"}
Jul 24 14:31:21.299696 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:21.299661    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:31:21.300003 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:21.299734    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:31:21.300003 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:21.299760    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:31:21.627368 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:21.627278    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq" event={"ID":"642f2c92-f994-4277-8613-d7371432db23","Type":"ContainerStarted","Data":"129297df54bdc69af57f8d651a29832cc5b917e3fca497bb9afef41c8d511b75"}
Jul 24 14:31:21.652329 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:21.652283    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq" podStartSLOduration=1.021184391 podStartE2EDuration="15.652269451s" totalImagesPullingTime="14.63108506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:31:21.650370406 +0000 UTC m=+538.435070871" watchObservedRunningTime="2026-07-24 14:31:21.652269451 +0000 UTC m=+538.436969915"
Jul 24 14:31:22.511461 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:22.511424    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:22.516163 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:22.516142    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:22.632198 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:22.632164    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:22.633062 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:22.633043    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq"
Jul 24 14:31:33.540089 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:33.540058    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-74vtf"
Jul 24 14:31:36.556773 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:36.556743    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j98hz"
Jul 24 14:31:48.481692 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.481648    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sbrlj"]
Jul 24 14:31:48.485099 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.485081    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:48.487700 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.487680    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 14:31:48.488850 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.488798    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vmp44\""
Jul 24 14:31:48.488976 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.488821    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 14:31:48.494556 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.494532    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sbrlj"]
Jul 24 14:31:48.558596 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.558570    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbmcj\" (UniqueName: \"kubernetes.io/projected/336ee837-69ab-4a7b-b2fa-0c070066c1d9-kube-api-access-bbmcj\") pod \"kuadrant-operator-catalog-sbrlj\" (UID: \"336ee837-69ab-4a7b-b2fa-0c070066c1d9\") " pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:48.659166 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.659137    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbmcj\" (UniqueName: \"kubernetes.io/projected/336ee837-69ab-4a7b-b2fa-0c070066c1d9-kube-api-access-bbmcj\") pod \"kuadrant-operator-catalog-sbrlj\" (UID: \"336ee837-69ab-4a7b-b2fa-0c070066c1d9\") " pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:48.668735 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.668712    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbmcj\" (UniqueName: \"kubernetes.io/projected/336ee837-69ab-4a7b-b2fa-0c070066c1d9-kube-api-access-bbmcj\") pod \"kuadrant-operator-catalog-sbrlj\" (UID: \"336ee837-69ab-4a7b-b2fa-0c070066c1d9\") " pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:48.795639 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.795613    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:48.836578 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.836553    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sbrlj"]
Jul 24 14:31:48.928294 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:48.928271    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sbrlj"]
Jul 24 14:31:48.930476 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:31:48.930448    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336ee837_69ab_4a7b_b2fa_0c070066c1d9.slice/crio-3b5dac0fb7c7534602db978364c0e810ab7793be77f7338fc9df7847165297b6 WatchSource:0}: Error finding container 3b5dac0fb7c7534602db978364c0e810ab7793be77f7338fc9df7847165297b6: Status 404 returned error can't find the container with id 3b5dac0fb7c7534602db978364c0e810ab7793be77f7338fc9df7847165297b6
Jul 24 14:31:49.046506 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.046439    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hsxvv"]
Jul 24 14:31:49.051013 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.050994    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:49.059254 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.059235    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hsxvv"]
Jul 24 14:31:49.163767 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.163739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cngth\" (UniqueName: \"kubernetes.io/projected/f9e220d2-b7d2-45c4-8021-94b9dd7b34d1-kube-api-access-cngth\") pod \"kuadrant-operator-catalog-hsxvv\" (UID: \"f9e220d2-b7d2-45c4-8021-94b9dd7b34d1\") " pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:49.264871 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.264821    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cngth\" (UniqueName: \"kubernetes.io/projected/f9e220d2-b7d2-45c4-8021-94b9dd7b34d1-kube-api-access-cngth\") pod \"kuadrant-operator-catalog-hsxvv\" (UID: \"f9e220d2-b7d2-45c4-8021-94b9dd7b34d1\") " pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:49.275670 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.275642    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cngth\" (UniqueName: \"kubernetes.io/projected/f9e220d2-b7d2-45c4-8021-94b9dd7b34d1-kube-api-access-cngth\") pod \"kuadrant-operator-catalog-hsxvv\" (UID: \"f9e220d2-b7d2-45c4-8021-94b9dd7b34d1\") " pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:49.360981 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.360914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:49.493601 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.493574    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hsxvv"]
Jul 24 14:31:49.513771 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:31:49.513740    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e220d2_b7d2_45c4_8021_94b9dd7b34d1.slice/crio-4ae74af431cdcaaa68e15ac233a79724be997f7785816036159dd09d71e72cd1 WatchSource:0}: Error finding container 4ae74af431cdcaaa68e15ac233a79724be997f7785816036159dd09d71e72cd1: Status 404 returned error can't find the container with id 4ae74af431cdcaaa68e15ac233a79724be997f7785816036159dd09d71e72cd1
Jul 24 14:31:49.757546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.757480    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj" event={"ID":"336ee837-69ab-4a7b-b2fa-0c070066c1d9","Type":"ContainerStarted","Data":"3b5dac0fb7c7534602db978364c0e810ab7793be77f7338fc9df7847165297b6"}
Jul 24 14:31:49.758683 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:49.758658    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv" event={"ID":"f9e220d2-b7d2-45c4-8021-94b9dd7b34d1","Type":"ContainerStarted","Data":"4ae74af431cdcaaa68e15ac233a79724be997f7785816036159dd09d71e72cd1"}
Jul 24 14:31:51.770394 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:51.770354    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj" event={"ID":"336ee837-69ab-4a7b-b2fa-0c070066c1d9","Type":"ContainerStarted","Data":"bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782"}
Jul 24 14:31:51.770394 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:51.770387    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj" podUID="336ee837-69ab-4a7b-b2fa-0c070066c1d9" containerName="registry-server" containerID="cri-o://bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782" gracePeriod=2
Jul 24 14:31:51.772201 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:51.772177    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv" event={"ID":"f9e220d2-b7d2-45c4-8021-94b9dd7b34d1","Type":"ContainerStarted","Data":"0116e09e835f24366b043bd866f8ff18fc142a426b999babfd1a437e29aab85e"}
Jul 24 14:31:51.790216 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:51.790174    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj" podStartSLOduration=1.783865238 podStartE2EDuration="3.790157957s" totalImagesPullingTime="2.006292719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:31:51.788189825 +0000 UTC m=+568.572890291" watchObservedRunningTime="2026-07-24 14:31:51.790157957 +0000 UTC m=+568.574858420"
Jul 24 14:31:51.805977 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:51.805935    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv" podStartSLOduration=1.379315524 podStartE2EDuration="2.805922229s" totalImagesPullingTime="1.426606705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:31:51.805614939 +0000 UTC m=+568.590315404" watchObservedRunningTime="2026-07-24 14:31:51.805922229 +0000 UTC m=+568.590622693"
Jul 24 14:31:52.015420 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.015400    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:52.091095 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.091040    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbmcj\" (UniqueName: \"kubernetes.io/projected/336ee837-69ab-4a7b-b2fa-0c070066c1d9-kube-api-access-bbmcj\") pod \"336ee837-69ab-4a7b-b2fa-0c070066c1d9\" (UID: \"336ee837-69ab-4a7b-b2fa-0c070066c1d9\") "
Jul 24 14:31:52.093104 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.093080    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336ee837-69ab-4a7b-b2fa-0c070066c1d9-kube-api-access-bbmcj" (OuterVolumeSpecName: "kube-api-access-bbmcj") pod "336ee837-69ab-4a7b-b2fa-0c070066c1d9" (UID: "336ee837-69ab-4a7b-b2fa-0c070066c1d9"). InnerVolumeSpecName "kube-api-access-bbmcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:31:52.191847 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.191819    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbmcj\" (UniqueName: \"kubernetes.io/projected/336ee837-69ab-4a7b-b2fa-0c070066c1d9-kube-api-access-bbmcj\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:31:52.777733 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.777702    2584 generic.go:356] "Generic (PLEG): container finished" podID="336ee837-69ab-4a7b-b2fa-0c070066c1d9" containerID="bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782" exitCode=0
Jul 24 14:31:52.778293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.777741    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj"
Jul 24 14:31:52.778293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.777788    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj" event={"ID":"336ee837-69ab-4a7b-b2fa-0c070066c1d9","Type":"ContainerDied","Data":"bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782"}
Jul 24 14:31:52.778293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.777865    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sbrlj" event={"ID":"336ee837-69ab-4a7b-b2fa-0c070066c1d9","Type":"ContainerDied","Data":"3b5dac0fb7c7534602db978364c0e810ab7793be77f7338fc9df7847165297b6"}
Jul 24 14:31:52.778293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.777900    2584 scope.go:117] "RemoveContainer" containerID="bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782"
Jul 24 14:31:52.786906 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.786892    2584 scope.go:117] "RemoveContainer" containerID="bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782"
Jul 24 14:31:52.787160 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:31:52.787144    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782\": container with ID starting with bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782 not found: ID does not exist" containerID="bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782"
Jul 24 14:31:52.787203 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.787169    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782"} err="failed to get container status \"bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782\": rpc error: code = NotFound desc = could not find container \"bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782\": container with ID starting with bea2b47ae56d13edeaeb27525e7fedba827828fbc4dd218bdc535317e2ec0782 not found: ID does not exist"
Jul 24 14:31:52.803972 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.803951    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sbrlj"]
Jul 24 14:31:52.811440 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:52.811418    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sbrlj"]
Jul 24 14:31:53.676483 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:53.676446    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="336ee837-69ab-4a7b-b2fa-0c070066c1d9" path="/var/lib/kubelet/pods/336ee837-69ab-4a7b-b2fa-0c070066c1d9/volumes"
Jul 24 14:31:59.361990 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:59.361954    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:59.362478 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:59.362021    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:59.383248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:59.383224    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:31:59.834139 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:31:59.834108    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hsxvv"
Jul 24 14:32:21.709679 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.709647    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"]
Jul 24 14:32:21.710057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.709981    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="336ee837-69ab-4a7b-b2fa-0c070066c1d9" containerName="registry-server"
Jul 24 14:32:21.710057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.709994    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="336ee837-69ab-4a7b-b2fa-0c070066c1d9" containerName="registry-server"
Jul 24 14:32:21.710138 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.710064    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="336ee837-69ab-4a7b-b2fa-0c070066c1d9" containerName="registry-server"
Jul 24 14:32:21.713106 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.713086    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:21.716276 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.716256    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wbqbf\""
Jul 24 14:32:21.725248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.725231    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"]
Jul 24 14:32:21.833994 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.833969    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:21.834154 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.834036    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rns26\" (UniqueName: \"kubernetes.io/projected/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-kube-api-access-rns26\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:21.934951 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.934921    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rns26\" (UniqueName: \"kubernetes.io/projected/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-kube-api-access-rns26\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:21.935093 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.934985    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:21.935339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.935320    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:21.945181 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:21.945160    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rns26\" (UniqueName: \"kubernetes.io/projected/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-kube-api-access-rns26\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:22.023312 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:22.023290    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:22.161131 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:22.161105    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"]
Jul 24 14:32:22.163896 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:32:22.163860    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d560c51_ab84_4e2f_acc8_3e6497c9e9b1.slice/crio-6ea1661a2b60feee7bc533ac42abd94f3f4d34c06a9bc17c3920a78f9f7253c4 WatchSource:0}: Error finding container 6ea1661a2b60feee7bc533ac42abd94f3f4d34c06a9bc17c3920a78f9f7253c4: Status 404 returned error can't find the container with id 6ea1661a2b60feee7bc533ac42abd94f3f4d34c06a9bc17c3920a78f9f7253c4
Jul 24 14:32:22.919569 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:22.919529    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh" event={"ID":"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1","Type":"ContainerStarted","Data":"6ea1661a2b60feee7bc533ac42abd94f3f4d34c06a9bc17c3920a78f9f7253c4"}
Jul 24 14:32:23.547626 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.547597    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:32:23.548180 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.548157    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:32:23.707969 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.707935    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"]
Jul 24 14:32:23.712764 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.712738    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:23.715598 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.715579    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 14:32:23.715737 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.715646    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qrwhq\""
Jul 24 14:32:23.720809 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.720787    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"]
Jul 24 14:32:23.849623 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.849544    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm5ss\" (UniqueName: \"kubernetes.io/projected/1b034609-fb4a-462a-ac66-825aa481033c-kube-api-access-pm5ss\") pod \"dns-operator-controller-manager-864dd78c4c-5jxjq\" (UID: \"1b034609-fb4a-462a-ac66-825aa481033c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:23.950633 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.950602    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm5ss\" (UniqueName: \"kubernetes.io/projected/1b034609-fb4a-462a-ac66-825aa481033c-kube-api-access-pm5ss\") pod \"dns-operator-controller-manager-864dd78c4c-5jxjq\" (UID: \"1b034609-fb4a-462a-ac66-825aa481033c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:23.961277 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:23.961251    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm5ss\" (UniqueName: \"kubernetes.io/projected/1b034609-fb4a-462a-ac66-825aa481033c-kube-api-access-pm5ss\") pod \"dns-operator-controller-manager-864dd78c4c-5jxjq\" (UID: \"1b034609-fb4a-462a-ac66-825aa481033c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:24.026328 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:24.026300    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:24.190814 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:24.190784    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"]
Jul 24 14:32:24.205506 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:32:24.205484    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b034609_fb4a_462a_ac66_825aa481033c.slice/crio-d9f10f9871375672021b30e75a9ce5227d89646477e12671fceabd9f50a6e019 WatchSource:0}: Error finding container d9f10f9871375672021b30e75a9ce5227d89646477e12671fceabd9f50a6e019: Status 404 returned error can't find the container with id d9f10f9871375672021b30e75a9ce5227d89646477e12671fceabd9f50a6e019
Jul 24 14:32:24.931958 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:24.931923    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq" event={"ID":"1b034609-fb4a-462a-ac66-825aa481033c","Type":"ContainerStarted","Data":"d9f10f9871375672021b30e75a9ce5227d89646477e12671fceabd9f50a6e019"}
Jul 24 14:32:26.544648 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.544614    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"]
Jul 24 14:32:26.548273 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.548256    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:26.550999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.550980    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pcxv7\""
Jul 24 14:32:26.558859 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.558806    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"]
Jul 24 14:32:26.673555 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.673420    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nkbs\" (UniqueName: \"kubernetes.io/projected/eee22f90-d20f-476e-a34c-f2432f7446e2-kube-api-access-4nkbs\") pod \"limitador-operator-controller-manager-859d7b78bd-xvlql\" (UID: \"eee22f90-d20f-476e-a34c-f2432f7446e2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:26.774951 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.774919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nkbs\" (UniqueName: \"kubernetes.io/projected/eee22f90-d20f-476e-a34c-f2432f7446e2-kube-api-access-4nkbs\") pod \"limitador-operator-controller-manager-859d7b78bd-xvlql\" (UID: \"eee22f90-d20f-476e-a34c-f2432f7446e2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:26.786376 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.786329    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nkbs\" (UniqueName: \"kubernetes.io/projected/eee22f90-d20f-476e-a34c-f2432f7446e2-kube-api-access-4nkbs\") pod \"limitador-operator-controller-manager-859d7b78bd-xvlql\" (UID: \"eee22f90-d20f-476e-a34c-f2432f7446e2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:26.859629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:26.859598    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:27.151609 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.151590    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"]
Jul 24 14:32:27.153043 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:32:27.153021    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee22f90_d20f_476e_a34c_f2432f7446e2.slice/crio-35fa17a772545d8374d3f3ee8d868a154c459f2df5226a5d5d48a70a1911ade7 WatchSource:0}: Error finding container 35fa17a772545d8374d3f3ee8d868a154c459f2df5226a5d5d48a70a1911ade7: Status 404 returned error can't find the container with id 35fa17a772545d8374d3f3ee8d868a154c459f2df5226a5d5d48a70a1911ade7
Jul 24 14:32:27.951062 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.951035    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq" event={"ID":"1b034609-fb4a-462a-ac66-825aa481033c","Type":"ContainerStarted","Data":"083b813703c55e961b08ee7a03054b720ceb9708f647bbe7d7a099d3f823aa65"}
Jul 24 14:32:27.951504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.951156    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:27.952411 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.952392    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql" event={"ID":"eee22f90-d20f-476e-a34c-f2432f7446e2","Type":"ContainerStarted","Data":"35fa17a772545d8374d3f3ee8d868a154c459f2df5226a5d5d48a70a1911ade7"}
Jul 24 14:32:27.953889 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.953868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh" event={"ID":"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1","Type":"ContainerStarted","Data":"441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca"}
Jul 24 14:32:27.953992 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.953970    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:27.971209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.971166    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq" podStartSLOduration=2.108771365 podStartE2EDuration="4.971152638s" totalImagesPullingTime="2.862381273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:27.969627267 +0000 UTC m=+604.754327726" watchObservedRunningTime="2026-07-24 14:32:27.971152638 +0000 UTC m=+604.755853102"
Jul 24 14:32:27.991194 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:27.991154    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh" podStartSLOduration=2.150627409 podStartE2EDuration="6.991140632s" totalImagesPullingTime="4.840513223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:27.990039051 +0000 UTC m=+604.774739516" watchObservedRunningTime="2026-07-24 14:32:27.991140632 +0000 UTC m=+604.775841095"
Jul 24 14:32:28.960904 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:28.960854    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql" event={"ID":"eee22f90-d20f-476e-a34c-f2432f7446e2","Type":"ContainerStarted","Data":"1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982"}
Jul 24 14:32:28.961343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:28.961043    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:28.992732 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:28.992643    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql" podStartSLOduration=1.422969533 podStartE2EDuration="2.992623437s" totalImagesPullingTime="1.569653904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:28.991847254 +0000 UTC m=+605.776547716" watchObservedRunningTime="2026-07-24 14:32:28.992623437 +0000 UTC m=+605.777323900"
Jul 24 14:32:38.963211 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:38.963180    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:38.963605 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:38.963266    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5jxjq"
Jul 24 14:32:39.968210 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:39.968182    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:40.961597 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.961563    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"]
Jul 24 14:32:40.961847 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.961806    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" containerName="manager" containerID="cri-o://441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca" gracePeriod=2
Jul 24 14:32:40.964067 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:40.964037    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:40.965721 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.965688    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"]
Jul 24 14:32:40.985526 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.985499    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"]
Jul 24 14:32:40.986410 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.986378    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" containerName="manager" containerID="cri-o://1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982" gracePeriod=2
Jul 24 14:32:40.988697 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:40.988663    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xvlql\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:40.990409 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.990381    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"]
Jul 24 14:32:40.990950 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:40.990921    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:40.997444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.997422    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"]
Jul 24 14:32:40.997975 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.997956    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" containerName="manager"
Jul 24 14:32:40.998042 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.997979    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" containerName="manager"
Jul 24 14:32:40.998042 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.998001    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" containerName="manager"
Jul 24 14:32:40.998042 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.998009    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" containerName="manager"
Jul 24 14:32:40.998152 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.998132    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" containerName="manager"
Jul 24 14:32:40.998152 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:40.998147    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" containerName="manager"
Jul 24 14:32:41.001360 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.001340    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.004669 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.004641    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xvlql\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:41.007110 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.007079    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:41.008589 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.008571    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"]
Jul 24 14:32:41.012549 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.012532    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"]
Jul 24 14:32:41.012743 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.012611    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:41.024586 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.024560    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"]
Jul 24 14:32:41.034034 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.034007    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:41.056278 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.056251    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xvlql\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:41.086552 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.086533    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5850421b-06de-47b8-a264-943554ff6655-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x2skt\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.086649 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.086567    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcklv\" (UniqueName: \"kubernetes.io/projected/f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d-kube-api-access-hcklv\") pod \"limitador-operator-controller-manager-859d7b78bd-vp9j7\" (UID: \"f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:41.086649 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.086628    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-928x6\" (UniqueName: \"kubernetes.io/projected/5850421b-06de-47b8-a264-943554ff6655-kube-api-access-928x6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x2skt\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.188113 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.188084    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5850421b-06de-47b8-a264-943554ff6655-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x2skt\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.188265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.188137    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcklv\" (UniqueName: \"kubernetes.io/projected/f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d-kube-api-access-hcklv\") pod \"limitador-operator-controller-manager-859d7b78bd-vp9j7\" (UID: \"f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:41.188265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.188179    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-928x6\" (UniqueName: \"kubernetes.io/projected/5850421b-06de-47b8-a264-943554ff6655-kube-api-access-928x6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x2skt\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.188527 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.188504    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5850421b-06de-47b8-a264-943554ff6655-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x2skt\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.205053 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.205027    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcklv\" (UniqueName: \"kubernetes.io/projected/f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d-kube-api-access-hcklv\") pod \"limitador-operator-controller-manager-859d7b78bd-vp9j7\" (UID: \"f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:41.208086 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.208054    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-928x6\" (UniqueName: \"kubernetes.io/projected/5850421b-06de-47b8-a264-943554ff6655-kube-api-access-928x6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x2skt\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.221420 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.221397    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"]
Jul 24 14:32:41.225726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.225709    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.228091 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.228067    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xvlql\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:41.229980 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.229958    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:41.239113 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.239091    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"]
Jul 24 14:32:41.242548 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.242531    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:41.245796 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.245780    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:41.252380 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.252354    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xvlql\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:41.254579 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:41.254558    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9rkxh\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:41.289129 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.289105    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-extensions-socket-volume\") pod \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") "
Jul 24 14:32:41.289227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.289155    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rns26\" (UniqueName: \"kubernetes.io/projected/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-kube-api-access-rns26\") pod \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\" (UID: \"5d560c51-ab84-4e2f-acc8-3e6497c9e9b1\") "
Jul 24 14:32:41.289227 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.289214    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nkbs\" (UniqueName: \"kubernetes.io/projected/eee22f90-d20f-476e-a34c-f2432f7446e2-kube-api-access-4nkbs\") pod \"eee22f90-d20f-476e-a34c-f2432f7446e2\" (UID: \"eee22f90-d20f-476e-a34c-f2432f7446e2\") "
Jul 24 14:32:41.289318 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.289302    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7zm4\" (UniqueName: \"kubernetes.io/projected/3218fc70-b704-42cb-85c7-c7193c6adb67-kube-api-access-c7zm4\") pod \"kuadrant-operator-controller-manager-86878c769f-5zhwn\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.289373 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.289340    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3218fc70-b704-42cb-85c7-c7193c6adb67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5zhwn\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.289574 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.289551    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" (UID: "5d560c51-ab84-4e2f-acc8-3e6497c9e9b1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:32:41.291250 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.291227    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-kube-api-access-rns26" (OuterVolumeSpecName: "kube-api-access-rns26") pod "5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" (UID: "5d560c51-ab84-4e2f-acc8-3e6497c9e9b1"). InnerVolumeSpecName "kube-api-access-rns26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:32:41.291571 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.291548    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee22f90-d20f-476e-a34c-f2432f7446e2-kube-api-access-4nkbs" (OuterVolumeSpecName: "kube-api-access-4nkbs") pod "eee22f90-d20f-476e-a34c-f2432f7446e2" (UID: "eee22f90-d20f-476e-a34c-f2432f7446e2"). InnerVolumeSpecName "kube-api-access-4nkbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:32:41.390455 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.390428    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7zm4\" (UniqueName: \"kubernetes.io/projected/3218fc70-b704-42cb-85c7-c7193c6adb67-kube-api-access-c7zm4\") pod \"kuadrant-operator-controller-manager-86878c769f-5zhwn\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.390572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.390490    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3218fc70-b704-42cb-85c7-c7193c6adb67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5zhwn\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.390620 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.390567    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-extensions-socket-volume\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:32:41.390620 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.390584    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rns26\" (UniqueName: \"kubernetes.io/projected/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1-kube-api-access-rns26\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:32:41.390620 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.390598    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nkbs\" (UniqueName: \"kubernetes.io/projected/eee22f90-d20f-476e-a34c-f2432f7446e2-kube-api-access-4nkbs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:32:41.390974 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.390956    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3218fc70-b704-42cb-85c7-c7193c6adb67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5zhwn\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.405822 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.405801    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7zm4\" (UniqueName: \"kubernetes.io/projected/3218fc70-b704-42cb-85c7-c7193c6adb67-kube-api-access-c7zm4\") pod \"kuadrant-operator-controller-manager-86878c769f-5zhwn\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.411589 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.411573    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:41.418614 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.418598    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:41.553371 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.553343    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:41.580923 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.580897    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"]
Jul 24 14:32:41.582784 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:32:41.582762    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5850421b_06de_47b8_a264_943554ff6655.slice/crio-a6d740c10cb82561ad37fe45daac35cf0a36803f3012b5951c1e9a9f11d0c907 WatchSource:0}: Error finding container a6d740c10cb82561ad37fe45daac35cf0a36803f3012b5951c1e9a9f11d0c907: Status 404 returned error can't find the container with id a6d740c10cb82561ad37fe45daac35cf0a36803f3012b5951c1e9a9f11d0c907
Jul 24 14:32:41.594339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.594316    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"]
Jul 24 14:32:41.596745 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:32:41.596718    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f47e37_2d8f_4485_a9f5_2cdb3c56fa5d.slice/crio-664e01fc6812e290a4f676e7397d36ccd94f8977f5729cf66ecd2bee5e5aa8b6 WatchSource:0}: Error finding container 664e01fc6812e290a4f676e7397d36ccd94f8977f5729cf66ecd2bee5e5aa8b6: Status 404 returned error can't find the container with id 664e01fc6812e290a4f676e7397d36ccd94f8977f5729cf66ecd2bee5e5aa8b6
Jul 24 14:32:41.672790 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.672760    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" path="/var/lib/kubelet/pods/5d560c51-ab84-4e2f-acc8-3e6497c9e9b1/volumes"
Jul 24 14:32:41.673158 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.673142    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee22f90-d20f-476e-a34c-f2432f7446e2" path="/var/lib/kubelet/pods/eee22f90-d20f-476e-a34c-f2432f7446e2/volumes"
Jul 24 14:32:41.706125 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:41.706105    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"]
Jul 24 14:32:41.708960 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:32:41.708928    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3218fc70_b704_42cb_85c7_c7193c6adb67.slice/crio-31b16049824a6f578a8ed3a21e0c9f08539508e24e195732e9687e056fcbeab4 WatchSource:0}: Error finding container 31b16049824a6f578a8ed3a21e0c9f08539508e24e195732e9687e056fcbeab4: Status 404 returned error can't find the container with id 31b16049824a6f578a8ed3a21e0c9f08539508e24e195732e9687e056fcbeab4
Jul 24 14:32:42.025138 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.025109    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn" event={"ID":"3218fc70-b704-42cb-85c7-c7193c6adb67","Type":"ContainerStarted","Data":"9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8"}
Jul 24 14:32:42.025552 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.025159    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn" event={"ID":"3218fc70-b704-42cb-85c7-c7193c6adb67","Type":"ContainerStarted","Data":"31b16049824a6f578a8ed3a21e0c9f08539508e24e195732e9687e056fcbeab4"}
Jul 24 14:32:42.025552 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.025281    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:42.026477 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.026456    2584 generic.go:356] "Generic (PLEG): container finished" podID="5d560c51-ab84-4e2f-acc8-3e6497c9e9b1" containerID="441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca" exitCode=0
Jul 24 14:32:42.026550 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.026493    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9rkxh"
Jul 24 14:32:42.026550 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.026544    2584 scope.go:117] "RemoveContainer" containerID="441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca"
Jul 24 14:32:42.028321 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.028245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7" event={"ID":"f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d","Type":"ContainerStarted","Data":"3dee103ee331eb5e61f5555f13ee0824c6922cce8a5887f5f8013a2199436f2a"}
Jul 24 14:32:42.028321 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.028291    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7" event={"ID":"f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d","Type":"ContainerStarted","Data":"664e01fc6812e290a4f676e7397d36ccd94f8977f5729cf66ecd2bee5e5aa8b6"}
Jul 24 14:32:42.029965 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.029941    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt" event={"ID":"5850421b-06de-47b8-a264-943554ff6655","Type":"ContainerStarted","Data":"f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058"}
Jul 24 14:32:42.030078 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.029978    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt" event={"ID":"5850421b-06de-47b8-a264-943554ff6655","Type":"ContainerStarted","Data":"a6d740c10cb82561ad37fe45daac35cf0a36803f3012b5951c1e9a9f11d0c907"}
Jul 24 14:32:42.030078 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.030052    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:42.031256 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.031233    2584 generic.go:356] "Generic (PLEG): container finished" podID="eee22f90-d20f-476e-a34c-f2432f7446e2" containerID="1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982" exitCode=0
Jul 24 14:32:42.031381 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.031262    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xvlql"
Jul 24 14:32:42.035518 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.035502    2584 scope.go:117] "RemoveContainer" containerID="441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca"
Jul 24 14:32:42.035766 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:42.035746    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca\": container with ID starting with 441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca not found: ID does not exist" containerID="441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca"
Jul 24 14:32:42.035884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.035771    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca"} err="failed to get container status \"441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca\": rpc error: code = NotFound desc = could not find container \"441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca\": container with ID starting with 441dd7ae757bb49a59ea7d15f0c6b415868494ee6a31d096a28646f13b331aca not found: ID does not exist"
Jul 24 14:32:42.035884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.035803    2584 scope.go:117] "RemoveContainer" containerID="1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982"
Jul 24 14:32:42.043060 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.043047    2584 scope.go:117] "RemoveContainer" containerID="1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982"
Jul 24 14:32:42.043277 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:42.043262    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982\": container with ID starting with 1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982 not found: ID does not exist" containerID="1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982"
Jul 24 14:32:42.043331 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.043283    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982"} err="failed to get container status \"1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982\": rpc error: code = NotFound desc = could not find container \"1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982\": container with ID starting with 1da16d38075a5a7ce9fc2d7595ffe9c525386e3d912a26bf8d38b386bae1d982 not found: ID does not exist"
Jul 24 14:32:42.065920 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.065879    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn" podStartSLOduration=1.065859765 podStartE2EDuration="1.065859765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:42.065381706 +0000 UTC m=+618.850082171" watchObservedRunningTime="2026-07-24 14:32:42.065859765 +0000 UTC m=+618.850560225"
Jul 24 14:32:42.112599 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.112556    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7" podStartSLOduration=2.11254205 podStartE2EDuration="2.11254205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:42.108480446 +0000 UTC m=+618.893180907" watchObservedRunningTime="2026-07-24 14:32:42.11254205 +0000 UTC m=+618.897242542"
Jul 24 14:32:42.139572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:42.139531    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt" podStartSLOduration=2.139516968 podStartE2EDuration="2.139516968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:32:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:32:42.13752156 +0000 UTC m=+618.922222024" watchObservedRunningTime="2026-07-24 14:32:42.139516968 +0000 UTC m=+618.924217431"
Jul 24 14:32:43.038096 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:43.038062    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:53.039769 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.039738    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:53.040235 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.039818    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:32:53.125586 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.125557    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"]
Jul 24 14:32:53.125821 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.125773    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt" podUID="5850421b-06de-47b8-a264-943554ff6655" containerName="manager" containerID="cri-o://f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058" gracePeriod=10
Jul 24 14:32:53.383305 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.383283    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:53.494682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.494642    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5850421b-06de-47b8-a264-943554ff6655-extensions-socket-volume\") pod \"5850421b-06de-47b8-a264-943554ff6655\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") "
Jul 24 14:32:53.494825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.494717    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-928x6\" (UniqueName: \"kubernetes.io/projected/5850421b-06de-47b8-a264-943554ff6655-kube-api-access-928x6\") pod \"5850421b-06de-47b8-a264-943554ff6655\" (UID: \"5850421b-06de-47b8-a264-943554ff6655\") "
Jul 24 14:32:53.495085 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.495059    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5850421b-06de-47b8-a264-943554ff6655-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5850421b-06de-47b8-a264-943554ff6655" (UID: "5850421b-06de-47b8-a264-943554ff6655"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:32:53.496574 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.496543    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5850421b-06de-47b8-a264-943554ff6655-kube-api-access-928x6" (OuterVolumeSpecName: "kube-api-access-928x6") pod "5850421b-06de-47b8-a264-943554ff6655" (UID: "5850421b-06de-47b8-a264-943554ff6655"). InnerVolumeSpecName "kube-api-access-928x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:32:53.595505 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.595436    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5850421b-06de-47b8-a264-943554ff6655-extensions-socket-volume\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:32:53.595505 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:53.595460    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-928x6\" (UniqueName: \"kubernetes.io/projected/5850421b-06de-47b8-a264-943554ff6655-kube-api-access-928x6\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:32:54.044661 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.044633    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp9j7"
Jul 24 14:32:54.094628 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.094594    2584 generic.go:356] "Generic (PLEG): container finished" podID="5850421b-06de-47b8-a264-943554ff6655" containerID="f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058" exitCode=0
Jul 24 14:32:54.094794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.094656    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt" event={"ID":"5850421b-06de-47b8-a264-943554ff6655","Type":"ContainerDied","Data":"f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058"}
Jul 24 14:32:54.094794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.094713    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt" event={"ID":"5850421b-06de-47b8-a264-943554ff6655","Type":"ContainerDied","Data":"a6d740c10cb82561ad37fe45daac35cf0a36803f3012b5951c1e9a9f11d0c907"}
Jul 24 14:32:54.094794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.094658    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"
Jul 24 14:32:54.094794 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.094743    2584 scope.go:117] "RemoveContainer" containerID="f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058"
Jul 24 14:32:54.104207 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.104187    2584 scope.go:117] "RemoveContainer" containerID="f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058"
Jul 24 14:32:54.104489 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:32:54.104466    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058\": container with ID starting with f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058 not found: ID does not exist" containerID="f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058"
Jul 24 14:32:54.104604 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.104499    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058"} err="failed to get container status \"f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058\": rpc error: code = NotFound desc = could not find container \"f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058\": container with ID starting with f10c7abb7d75fe766107fa309213067a07cd7835a09c17b6e1e8c2b50736b058 not found: ID does not exist"
Jul 24 14:32:54.119533 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.119511    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"]
Jul 24 14:32:54.125122 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:54.125095    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x2skt"]
Jul 24 14:32:55.671077 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:32:55.671035    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5850421b-06de-47b8-a264-943554ff6655" path="/var/lib/kubelet/pods/5850421b-06de-47b8-a264-943554ff6655/volumes"
Jul 24 14:33:09.623803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.623763    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"]
Jul 24 14:33:09.624378 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.624362    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5850421b-06de-47b8-a264-943554ff6655" containerName="manager"
Jul 24 14:33:09.624442 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.624381    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5850421b-06de-47b8-a264-943554ff6655" containerName="manager"
Jul 24 14:33:09.624484 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.624464    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5850421b-06de-47b8-a264-943554ff6655" containerName="manager"
Jul 24 14:33:09.628670 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.628649    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.632258 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.632240    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hrwjz\""
Jul 24 14:33:09.654776 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.654753    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"]
Jul 24 14:33:09.732251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732220    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732440 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732272    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732440 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732373    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp6g\" (UniqueName: \"kubernetes.io/projected/3c712d07-40d1-4f63-837f-26a2ee2dc27f-kube-api-access-4lp6g\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732440 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732410    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732579 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732463    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732579 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732480    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732579 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732499    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732579 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732522    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.732707 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.732603    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.833818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.833781    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834010 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.833867    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp6g\" (UniqueName: \"kubernetes.io/projected/3c712d07-40d1-4f63-837f-26a2ee2dc27f-kube-api-access-4lp6g\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834010 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.833899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834010 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.833934    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834010 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.833957    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834010 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.833981    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834022    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834083    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834114    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834493 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834470    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834493 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834456    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834608 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834552    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834777    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.834978 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.834954    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.836575 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.836550    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.836679 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.836659    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.843973 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.843950    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c712d07-40d1-4f63-837f-26a2ee2dc27f-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.844095 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.844076    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp6g\" (UniqueName: \"kubernetes.io/projected/3c712d07-40d1-4f63-837f-26a2ee2dc27f-kube-api-access-4lp6g\") pod \"maas-default-gateway-openshift-default-78bb78ff78-7dwxz\" (UID: \"3c712d07-40d1-4f63-837f-26a2ee2dc27f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:09.939343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:09.939264    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:10.097073 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:10.097037    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"]
Jul 24 14:33:10.098591 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:33:10.098565    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c712d07_40d1_4f63_837f_26a2ee2dc27f.slice/crio-8377d99d725dedad22291a633f1e75e67b9b3eb091ad1a6d814eb97727d2fb4d WatchSource:0}: Error finding container 8377d99d725dedad22291a633f1e75e67b9b3eb091ad1a6d814eb97727d2fb4d: Status 404 returned error can't find the container with id 8377d99d725dedad22291a633f1e75e67b9b3eb091ad1a6d814eb97727d2fb4d
Jul 24 14:33:10.100716 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:10.100686    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:33:10.100816 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:10.100761    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:33:10.100816 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:10.100803    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:33:10.176595 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:10.176571    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" event={"ID":"3c712d07-40d1-4f63-837f-26a2ee2dc27f","Type":"ContainerStarted","Data":"8377d99d725dedad22291a633f1e75e67b9b3eb091ad1a6d814eb97727d2fb4d"}
Jul 24 14:33:11.182746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:11.182718    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" event={"ID":"3c712d07-40d1-4f63-837f-26a2ee2dc27f","Type":"ContainerStarted","Data":"8457a278723b04fef56b766f6ff81e815a29fd3e212db3b5655bb38d24d5bf74"}
Jul 24 14:33:11.207752 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:11.207709    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" podStartSLOduration=2.207693699 podStartE2EDuration="2.207693699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:11.206180592 +0000 UTC m=+647.990881056" watchObservedRunningTime="2026-07-24 14:33:11.207693699 +0000 UTC m=+647.992394163"
Jul 24 14:33:11.939461 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:11.939424    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:11.944196 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:11.944171    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:12.188424 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:12.188383    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:12.189198 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:12.189178    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:33:12.977484 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:12.977457    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:33:12.981825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:12.981806    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:12.985083 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:12.985063    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wqjkb\""
Jul 24 14:33:12.985256 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:12.985206    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 14:33:13.041515 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.041488    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:33:13.061888 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.061866    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xf8b\" (UniqueName: \"kubernetes.io/projected/3968abd3-b4b5-4f9e-8c74-b45738a10c5d-kube-api-access-5xf8b\") pod \"limitador-limitador-54d6d988d5-qhrnr\" (UID: \"3968abd3-b4b5-4f9e-8c74-b45738a10c5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.061980 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.061935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3968abd3-b4b5-4f9e-8c74-b45738a10c5d-config-file\") pod \"limitador-limitador-54d6d988d5-qhrnr\" (UID: \"3968abd3-b4b5-4f9e-8c74-b45738a10c5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.076218 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.076189    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:33:13.163164 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.163140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xf8b\" (UniqueName: \"kubernetes.io/projected/3968abd3-b4b5-4f9e-8c74-b45738a10c5d-kube-api-access-5xf8b\") pod \"limitador-limitador-54d6d988d5-qhrnr\" (UID: \"3968abd3-b4b5-4f9e-8c74-b45738a10c5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.163326 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.163204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3968abd3-b4b5-4f9e-8c74-b45738a10c5d-config-file\") pod \"limitador-limitador-54d6d988d5-qhrnr\" (UID: \"3968abd3-b4b5-4f9e-8c74-b45738a10c5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.163769 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.163750    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3968abd3-b4b5-4f9e-8c74-b45738a10c5d-config-file\") pod \"limitador-limitador-54d6d988d5-qhrnr\" (UID: \"3968abd3-b4b5-4f9e-8c74-b45738a10c5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.180560 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.180536    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xf8b\" (UniqueName: \"kubernetes.io/projected/3968abd3-b4b5-4f9e-8c74-b45738a10c5d-kube-api-access-5xf8b\") pod \"limitador-limitador-54d6d988d5-qhrnr\" (UID: \"3968abd3-b4b5-4f9e-8c74-b45738a10c5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.293077 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.293051    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:13.438202 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:13.438172    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:33:13.442377 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:33:13.442353    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3968abd3_b4b5_4f9e_8c74_b45738a10c5d.slice/crio-87cbebc89a1f59f872d55294a75606af5805b612fede3789e979980886891f5c WatchSource:0}: Error finding container 87cbebc89a1f59f872d55294a75606af5805b612fede3789e979980886891f5c: Status 404 returned error can't find the container with id 87cbebc89a1f59f872d55294a75606af5805b612fede3789e979980886891f5c
Jul 24 14:33:14.201253 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:14.201219    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr" event={"ID":"3968abd3-b4b5-4f9e-8c74-b45738a10c5d","Type":"ContainerStarted","Data":"87cbebc89a1f59f872d55294a75606af5805b612fede3789e979980886891f5c"}
Jul 24 14:33:16.216405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:16.216324    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr" event={"ID":"3968abd3-b4b5-4f9e-8c74-b45738a10c5d","Type":"ContainerStarted","Data":"9b2fefbb82c428d22713d9b118ce049569fcd81eba0ec4f52eb9401758969e20"}
Jul 24 14:33:16.216405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:16.216380    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:33:16.237554 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:16.237512    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr" podStartSLOduration=1.856778084 podStartE2EDuration="4.23749716s" totalImagesPullingTime="2.380719076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:33:16.236238268 +0000 UTC m=+653.020938732" watchObservedRunningTime="2026-07-24 14:33:16.23749716 +0000 UTC m=+653.022197625"
Jul 24 14:33:27.222873 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:33:27.222825    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"
Jul 24 14:34:25.193433 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:25.193359    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:34:32.584055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:32.584029    2584 generic.go:356] "Generic (PLEG): container finished" podID="3c712d07-40d1-4f63-837f-26a2ee2dc27f" containerID="8457a278723b04fef56b766f6ff81e815a29fd3e212db3b5655bb38d24d5bf74" exitCode=0
Jul 24 14:34:32.584383 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:32.584093    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" event={"ID":"3c712d07-40d1-4f63-837f-26a2ee2dc27f","Type":"ContainerDied","Data":"8457a278723b04fef56b766f6ff81e815a29fd3e212db3b5655bb38d24d5bf74"}
Jul 24 14:34:32.584453 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:32.584439    2584 scope.go:117] "RemoveContainer" containerID="8457a278723b04fef56b766f6ff81e815a29fd3e212db3b5655bb38d24d5bf74"
Jul 24 14:34:32.585355 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:32.585330    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:34:32.585406 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:32.585393    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:34:32.585444 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:32.585427    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:34:33.591532 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:33.591500    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" event={"ID":"3c712d07-40d1-4f63-837f-26a2ee2dc27f","Type":"ContainerStarted","Data":"e195bb11f9e96b4aa8eca06d7799d3755048f4c84deb79fba91b42e8b920cba9"}
Jul 24 14:34:33.939857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:33.939739    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:34:33.940005 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:33.939961    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7dwxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 14:34:33.940052 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:33.940007    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" podUID="3c712d07-40d1-4f63-837f-26a2ee2dc27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 14:34:34.940013 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:34.939971    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7dwxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 14:34:34.940460 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:34.940031    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" podUID="3c712d07-40d1-4f63-837f-26a2ee2dc27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 14:34:35.939925 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:35.939891    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7dwxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 14:34:35.940112 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:35.939948    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" podUID="3c712d07-40d1-4f63-837f-26a2ee2dc27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 14:34:36.940079 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:36.940041    2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-7dwxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 24 14:34:36.940230 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:36.940098    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz" podUID="3c712d07-40d1-4f63-837f-26a2ee2dc27f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 24 14:34:37.943462 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:37.943435    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:34:37.943967 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:37.943827    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:34:37.944359 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:37.944340    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-7dwxz"
Jul 24 14:34:52.948238 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:34:52.948208    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:35:14.045754 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:35:14.045711    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:35:18.466411 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:35:18.466378    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:35:30.946392 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:35:30.946358    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:35:40.953061 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:35:40.953022    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:36:56.160663 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:36:56.160632    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:37:07.656129 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:37:07.656094    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:37:17.952290 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:37:17.952206    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:37:23.581575 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:37:23.581548    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:37:23.583123 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:37:23.583098    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:38:59.656169 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:38:59.656137    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:39:10.766922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:39:10.766885    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:39:36.954424 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:39:36.954386    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:39:45.971593 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:39:45.971556    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:39:54.566261 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:39:54.566230    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:40:03.974530 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:40:03.974500    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:40:14.167030 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:40:14.166996    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:40:19.975117 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:40:19.975027    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:40:31.168916 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:40:31.168813    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:40:40.992580 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:40:40.992541    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:40:52.404129 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:40:52.404096    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:41:02.764660 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:41:02.764620    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:41:24.787905 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:41:24.787872    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:41:35.594998 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:41:35.594960    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:42:02.789521 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:02.789491    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:42:16.872035 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:16.872000    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:42:23.612103 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:23.612065    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:42:23.613121 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:23.613102    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:42:32.056156 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:32.056124    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:42:40.866715 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:40.866682    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:42:50.166344 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:50.166310    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:42:59.364935 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:42:59.364898    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:43:08.861515 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:08.861483    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:43:36.326823 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:36.326746    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"]
Jul 24 14:43:36.327343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:36.327001    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn" podUID="3218fc70-b704-42cb-85c7-c7193c6adb67" containerName="manager" containerID="cri-o://9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8" gracePeriod=10
Jul 24 14:43:39.681743 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.681719    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:43:39.880004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.879968    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7zm4\" (UniqueName: \"kubernetes.io/projected/3218fc70-b704-42cb-85c7-c7193c6adb67-kube-api-access-c7zm4\") pod \"3218fc70-b704-42cb-85c7-c7193c6adb67\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") "
Jul 24 14:43:39.880168 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.880046    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3218fc70-b704-42cb-85c7-c7193c6adb67-extensions-socket-volume\") pod \"3218fc70-b704-42cb-85c7-c7193c6adb67\" (UID: \"3218fc70-b704-42cb-85c7-c7193c6adb67\") "
Jul 24 14:43:39.880432 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.880400    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3218fc70-b704-42cb-85c7-c7193c6adb67-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3218fc70-b704-42cb-85c7-c7193c6adb67" (UID: "3218fc70-b704-42cb-85c7-c7193c6adb67"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:43:39.881902 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.881871    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3218fc70-b704-42cb-85c7-c7193c6adb67-kube-api-access-c7zm4" (OuterVolumeSpecName: "kube-api-access-c7zm4") pod "3218fc70-b704-42cb-85c7-c7193c6adb67" (UID: "3218fc70-b704-42cb-85c7-c7193c6adb67"). InnerVolumeSpecName "kube-api-access-c7zm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:43:39.980983 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.980957    2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3218fc70-b704-42cb-85c7-c7193c6adb67-extensions-socket-volume\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:43:39.980983 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:39.980978    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7zm4\" (UniqueName: \"kubernetes.io/projected/3218fc70-b704-42cb-85c7-c7193c6adb67-kube-api-access-c7zm4\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:43:40.218004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.217911    2584 generic.go:356] "Generic (PLEG): container finished" podID="3218fc70-b704-42cb-85c7-c7193c6adb67" containerID="9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8" exitCode=0
Jul 24 14:43:40.218004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.217970    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"
Jul 24 14:43:40.218004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.217979    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn" event={"ID":"3218fc70-b704-42cb-85c7-c7193c6adb67","Type":"ContainerDied","Data":"9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8"}
Jul 24 14:43:40.218263 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.218063    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn" event={"ID":"3218fc70-b704-42cb-85c7-c7193c6adb67","Type":"ContainerDied","Data":"31b16049824a6f578a8ed3a21e0c9f08539508e24e195732e9687e056fcbeab4"}
Jul 24 14:43:40.218263 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.218092    2584 scope.go:117] "RemoveContainer" containerID="9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8"
Jul 24 14:43:40.227962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.227947    2584 scope.go:117] "RemoveContainer" containerID="9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8"
Jul 24 14:43:40.228196 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:43:40.228180    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8\": container with ID starting with 9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8 not found: ID does not exist" containerID="9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8"
Jul 24 14:43:40.228258 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.228207    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8"} err="failed to get container status \"9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8\": rpc error: code = NotFound desc = could not find container \"9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8\": container with ID starting with 9a5dc735a80b56422bf8620266ab5816228ae6bea97cf2745235e69386c29cb8 not found: ID does not exist"
Jul 24 14:43:40.252361 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.252337    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"]
Jul 24 14:43:40.256694 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:40.256674    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5zhwn"]
Jul 24 14:43:41.670269 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:43:41.670237    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3218fc70-b704-42cb-85c7-c7193c6adb67" path="/var/lib/kubelet/pods/3218fc70-b704-42cb-85c7-c7193c6adb67/volumes"
Jul 24 14:44:37.607744 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.607713    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"]
Jul 24 14:44:37.610240 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.608266    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3218fc70-b704-42cb-85c7-c7193c6adb67" containerName="manager"
Jul 24 14:44:37.610240 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.608284    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3218fc70-b704-42cb-85c7-c7193c6adb67" containerName="manager"
Jul 24 14:44:37.610240 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.608369    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3218fc70-b704-42cb-85c7-c7193c6adb67" containerName="manager"
Jul 24 14:44:37.611369 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.611352    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.614111 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.614092    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wbqbf\""
Jul 24 14:44:37.623575 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.623557    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"]
Jul 24 14:44:37.670971 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.670948    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnz9g\" (UniqueName: \"kubernetes.io/projected/246d5a54-7d04-4a20-ae5c-fa66c1e96d16-kube-api-access-xnz9g\") pod \"kuadrant-operator-controller-manager-86878c769f-nz76j\" (UID: \"246d5a54-7d04-4a20-ae5c-fa66c1e96d16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.671100 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.671001    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/246d5a54-7d04-4a20-ae5c-fa66c1e96d16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nz76j\" (UID: \"246d5a54-7d04-4a20-ae5c-fa66c1e96d16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.771682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.771657    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnz9g\" (UniqueName: \"kubernetes.io/projected/246d5a54-7d04-4a20-ae5c-fa66c1e96d16-kube-api-access-xnz9g\") pod \"kuadrant-operator-controller-manager-86878c769f-nz76j\" (UID: \"246d5a54-7d04-4a20-ae5c-fa66c1e96d16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.771803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.771713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/246d5a54-7d04-4a20-ae5c-fa66c1e96d16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nz76j\" (UID: \"246d5a54-7d04-4a20-ae5c-fa66c1e96d16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.772098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.772078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/246d5a54-7d04-4a20-ae5c-fa66c1e96d16-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nz76j\" (UID: \"246d5a54-7d04-4a20-ae5c-fa66c1e96d16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.784209 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.784188    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnz9g\" (UniqueName: \"kubernetes.io/projected/246d5a54-7d04-4a20-ae5c-fa66c1e96d16-kube-api-access-xnz9g\") pod \"kuadrant-operator-controller-manager-86878c769f-nz76j\" (UID: \"246d5a54-7d04-4a20-ae5c-fa66c1e96d16\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:37.922275 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:37.922212    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:38.064064 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:38.064031    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"]
Jul 24 14:44:38.073709 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:44:38.073654    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246d5a54_7d04_4a20_ae5c_fa66c1e96d16.slice/crio-52b9249bff0afaa62850fb2fb2e31ac86e0d4f77c5516f99bf5745be95fff9a4 WatchSource:0}: Error finding container 52b9249bff0afaa62850fb2fb2e31ac86e0d4f77c5516f99bf5745be95fff9a4: Status 404 returned error can't find the container with id 52b9249bff0afaa62850fb2fb2e31ac86e0d4f77c5516f99bf5745be95fff9a4
Jul 24 14:44:38.502790 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:38.502755    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j" event={"ID":"246d5a54-7d04-4a20-ae5c-fa66c1e96d16","Type":"ContainerStarted","Data":"a6a4055deccc3964d859cfec8abf5a7f1be9f97cafe97e6eb6ab6b96ca17b1db"}
Jul 24 14:44:38.503001 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:38.502803    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j" event={"ID":"246d5a54-7d04-4a20-ae5c-fa66c1e96d16","Type":"ContainerStarted","Data":"52b9249bff0afaa62850fb2fb2e31ac86e0d4f77c5516f99bf5745be95fff9a4"}
Jul 24 14:44:38.503001 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:38.502866    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:38.523519 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:38.523481    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j" podStartSLOduration=1.523466869 podStartE2EDuration="1.523466869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:44:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:44:38.52292076 +0000 UTC m=+1335.307621223" watchObservedRunningTime="2026-07-24 14:44:38.523466869 +0000 UTC m=+1335.308167332"
Jul 24 14:44:49.510427 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:49.510398    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz76j"
Jul 24 14:44:58.056728 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:44:58.056694    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:45:00.154405 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.154378    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"]
Jul 24 14:45:00.157802 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.157786    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"
Jul 24 14:45:00.160659 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.160629    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:45:00.164413 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.164373    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"]
Jul 24 14:45:00.168178 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.168157    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"
Jul 24 14:45:00.508456 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.508436    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"]
Jul 24 14:45:00.510426 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:45:00.510401    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714c95c2_0de5_4c6e_8619_4b653035235d.slice/crio-5a702903a7d9718c39cbcb12844b99de87229c41aa5d2e7a783e4008a6398263 WatchSource:0}: Error finding container 5a702903a7d9718c39cbcb12844b99de87229c41aa5d2e7a783e4008a6398263: Status 404 returned error can't find the container with id 5a702903a7d9718c39cbcb12844b99de87229c41aa5d2e7a783e4008a6398263
Jul 24 14:45:00.512228 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.512213    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 14:45:00.612670 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:00.612643    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp" event={"ID":"714c95c2-0de5-4c6e-8619-4b653035235d","Type":"ContainerStarted","Data":"5a702903a7d9718c39cbcb12844b99de87229c41aa5d2e7a783e4008a6398263"}
Jul 24 14:45:03.628559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:03.628527    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp" event={"ID":"714c95c2-0de5-4c6e-8619-4b653035235d","Type":"ContainerStarted","Data":"1b41d34590ffc0676242f86c7f3b4761ef95d7b0be7530ac5932ab68fd458a6f"}
Jul 24 14:45:03.650395 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:03.650350    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp" podStartSLOduration=1.218482405 podStartE2EDuration="3.650335672s" totalImagesPullingTime="2.431853267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:45:03.648526309 +0000 UTC m=+1360.433226771" watchObservedRunningTime="2026-07-24 14:45:03.650335672 +0000 UTC m=+1360.435036135"
Jul 24 14:45:04.635497 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:04.635461    2584 generic.go:356] "Generic (PLEG): container finished" podID="714c95c2-0de5-4c6e-8619-4b653035235d" containerID="1b41d34590ffc0676242f86c7f3b4761ef95d7b0be7530ac5932ab68fd458a6f" exitCode=0
Jul 24 14:45:04.635888 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:04.635536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp" event={"ID":"714c95c2-0de5-4c6e-8619-4b653035235d","Type":"ContainerDied","Data":"1b41d34590ffc0676242f86c7f3b4761ef95d7b0be7530ac5932ab68fd458a6f"}
Jul 24 14:45:05.778036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:05.778011    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"
Jul 24 14:45:06.646258 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:06.646229    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp"
Jul 24 14:45:06.646258 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:06.646254    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748405-gxphp" event={"ID":"714c95c2-0de5-4c6e-8619-4b653035235d","Type":"ContainerDied","Data":"5a702903a7d9718c39cbcb12844b99de87229c41aa5d2e7a783e4008a6398263"}
Jul 24 14:45:06.646468 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:06.646298    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a702903a7d9718c39cbcb12844b99de87229c41aa5d2e7a783e4008a6398263"
Jul 24 14:45:09.057993 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:09.057962    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:45:17.762163 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:17.762124    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:45:28.350299 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:28.350266    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:45:38.154746 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:38.150319    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:45:49.564146 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:49.564112    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:45:59.360513 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:45:59.360478    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:46:10.150421 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:46:10.150385    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:46:20.453105 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:46:20.453017    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:46:30.544776 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:46:30.544747    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:46:40.955116 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:46:40.955081    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:47:02.661029 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:02.660994    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:47:12.752548 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:12.752516    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:47:23.647126 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:23.647100    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:47:23.648952 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:23.648933    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:47:26.046989 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:26.046960    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:47:43.181587 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:43.181544    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:47:56.946799 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:47:56.946753    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:48:04.253873 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:48:04.253828    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:48:18.552103 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:48:18.552065    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:48:27.766417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:48:27.766380    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:48:37.070096 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:48:37.070056    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:48:46.988851 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:48:46.988810    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:48:55.671979 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:48:55.671947    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:49:05.052229 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:49:05.052190    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:49:19.368351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:49:19.368264    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:49:28.257248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:49:28.257218    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:49:37.961112 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:49:37.961064    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:49:49.850174 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:49:49.850141    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:01.648970 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:01.648938    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:10.359303 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:10.359267    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:21.672106 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:21.672079    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:30.656580 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:30.656539    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:39.856531 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:39.856490    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:47.364729 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:47.364651    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:50:56.252485 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:50:56.252448    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:51:17.660649 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:17.660620    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:51:25.205674 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.205641    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"]
Jul 24 14:51:25.206072 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.206057    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714c95c2-0de5-4c6e-8619-4b653035235d" containerName="cleanup"
Jul 24 14:51:25.206072 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.206069    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="714c95c2-0de5-4c6e-8619-4b653035235d" containerName="cleanup"
Jul 24 14:51:25.206160 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.206149    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="714c95c2-0de5-4c6e-8619-4b653035235d" containerName="cleanup"
Jul 24 14:51:25.209267 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.209252    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.215258 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.215237    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2e2d5bdd-openshift-default-dockercfg-qz59l\""
Jul 24 14:51:25.245260 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.245243    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"]
Jul 24 14:51:25.368162 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368131    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdnjp\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-kube-api-access-xdnjp\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368347 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-token\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368347 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368209    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-data\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368347 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368300    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-podinfo\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368347 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368338    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-socket\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368355    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-certs\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368370    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-envoy\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368423    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-credential-socket\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.368551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.368457    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istiod-ca-cert\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469144 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469063    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-credential-socket\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469144 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469098    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istiod-ca-cert\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469144 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469137    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdnjp\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-kube-api-access-xdnjp\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469162    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-token\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469181    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-data\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469246    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-podinfo\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469282    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-socket\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-certs\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469326    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-envoy\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469828 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469509    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-credential-socket\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469828 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469673    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-certs\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469828 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469732    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-socket\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.469828 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469802    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-data\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.470137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.469982    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istiod-ca-cert\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.471490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.471473    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-envoy\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.471769 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.471752    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-podinfo\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.478567 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.478545    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-token\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.479112 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.479092    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdnjp\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-kube-api-access-xdnjp\") pod \"e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") " pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.521173 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.521144    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:25.668740 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:51:25.668708    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bded7fa_3eda_447a_a56f_d5b0eb96aa92.slice/crio-dd9f9ab3bdb8b7bfdeeb70651cdc016119cb967b9c2286081d33888eaa239cc5 WatchSource:0}: Error finding container dd9f9ab3bdb8b7bfdeeb70651cdc016119cb967b9c2286081d33888eaa239cc5: Status 404 returned error can't find the container with id dd9f9ab3bdb8b7bfdeeb70651cdc016119cb967b9c2286081d33888eaa239cc5
Jul 24 14:51:25.670807 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.670770    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:51:25.670926 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.670850    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:51:25.670926 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.670878    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:51:25.672416 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:25.672392    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"]
Jul 24 14:51:26.497582 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:26.497550    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" event={"ID":"4bded7fa-3eda-447a-a56f-d5b0eb96aa92","Type":"ContainerStarted","Data":"bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"}
Jul 24 14:51:26.498046 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:26.497597    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" event={"ID":"4bded7fa-3eda-447a-a56f-d5b0eb96aa92","Type":"ContainerStarted","Data":"dd9f9ab3bdb8b7bfdeeb70651cdc016119cb967b9c2286081d33888eaa239cc5"}
Jul 24 14:51:26.522189 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:26.522162    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:26.525354 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:26.525300    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" podStartSLOduration=1.5252818110000002 podStartE2EDuration="1.525281811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:51:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:51:26.524109645 +0000 UTC m=+1743.308810108" watchObservedRunningTime="2026-07-24 14:51:26.525281811 +0000 UTC m=+1743.309982275"
Jul 24 14:51:26.527677 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:26.527657    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:27.502416 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:27.502387    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:27.503242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:27.503225    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:30.796267 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:30.796234    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:51:38.556366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:38.556335    2584 generic.go:356] "Generic (PLEG): container finished" podID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerID="bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458" exitCode=0
Jul 24 14:51:38.556765 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:38.556376    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" event={"ID":"4bded7fa-3eda-447a-a56f-d5b0eb96aa92","Type":"ContainerDied","Data":"bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"}
Jul 24 14:51:38.556765 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:38.556723    2584 scope.go:117] "RemoveContainer" containerID="bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"
Jul 24 14:51:38.557667 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:38.557642    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:51:38.557752 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:38.557706    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:51:38.557752 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:38.557733    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:51:39.563082 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:39.563047    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" event={"ID":"4bded7fa-3eda-447a-a56f-d5b0eb96aa92","Type":"ContainerStarted","Data":"4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13"}
Jul 24 14:51:40.521626 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:40.521591    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:40.522811 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:40.522784    2584 patch_prober.go:28] interesting pod/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 24 14:51:40.522956 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:40.522850    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 24 14:51:41.521800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:41.521761    2584 patch_prober.go:28] interesting pod/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 24 14:51:41.522190 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:41.521817    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 24 14:51:42.502589 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:42.502552    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:42.521892 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:42.521860    2584 patch_prober.go:28] interesting pod/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 24 14:51:42.522219 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:42.521913    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 24 14:51:43.525968 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:43.525942    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:43.526958 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:43.526938    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:51:53.691962 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.691931    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"]
Jul 24 14:51:53.695577 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.695561    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:53.704446 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.704429    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:51:53.704567 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.704552    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:51:53.707959 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.707934    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"]
Jul 24 14:51:53.723921 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.723898    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca074a3-3b95-4cc9-991b-41b03879b8d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm\" (UID: \"4ca074a3-3b95-4cc9-991b-41b03879b8d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:53.824664 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.824634    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca074a3-3b95-4cc9-991b-41b03879b8d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm\" (UID: \"4ca074a3-3b95-4cc9-991b-41b03879b8d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:53.825257 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:53.825239    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca074a3-3b95-4cc9-991b-41b03879b8d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm\" (UID: \"4ca074a3-3b95-4cc9-991b-41b03879b8d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:54.005469 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:54.005400    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:54.145384 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:54.145343    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"]
Jul 24 14:51:54.149764 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:51:54.149739    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca074a3_3b95_4cc9_991b_41b03879b8d8.slice/crio-876bd4a0de0fce439b13672e4a2bdd087dd896a0650112d3b311039c601f1a15 WatchSource:0}: Error finding container 876bd4a0de0fce439b13672e4a2bdd087dd896a0650112d3b311039c601f1a15: Status 404 returned error can't find the container with id 876bd4a0de0fce439b13672e4a2bdd087dd896a0650112d3b311039c601f1a15
Jul 24 14:51:54.647806 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:54.647726    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm" event={"ID":"4ca074a3-3b95-4cc9-991b-41b03879b8d8","Type":"ContainerStarted","Data":"21a085ce0d9a4978fef5cdf1e9553d9093d42a70d96c0d30e490fdd948bf6f1f"}
Jul 24 14:51:54.647806 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:54.647779    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm" event={"ID":"4ca074a3-3b95-4cc9-991b-41b03879b8d8","Type":"ContainerStarted","Data":"876bd4a0de0fce439b13672e4a2bdd087dd896a0650112d3b311039c601f1a15"}
Jul 24 14:51:54.675743 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:54.675699    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm" podStartSLOduration=1.6756838969999999 podStartE2EDuration="1.675683897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:51:54.674161311 +0000 UTC m=+1771.458861775" watchObservedRunningTime="2026-07-24 14:51:54.675683897 +0000 UTC m=+1771.460384360"
Jul 24 14:51:55.655339 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:55.655307    2584 generic.go:356] "Generic (PLEG): container finished" podID="4ca074a3-3b95-4cc9-991b-41b03879b8d8" containerID="21a085ce0d9a4978fef5cdf1e9553d9093d42a70d96c0d30e490fdd948bf6f1f" exitCode=0
Jul 24 14:51:55.655697 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:55.655369    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm" event={"ID":"4ca074a3-3b95-4cc9-991b-41b03879b8d8","Type":"ContainerDied","Data":"21a085ce0d9a4978fef5cdf1e9553d9093d42a70d96c0d30e490fdd948bf6f1f"}
Jul 24 14:51:56.788479 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:56.788450    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:56.849842 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:56.849813    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca074a3-3b95-4cc9-991b-41b03879b8d8-maas-api-service-ca\") pod \"4ca074a3-3b95-4cc9-991b-41b03879b8d8\" (UID: \"4ca074a3-3b95-4cc9-991b-41b03879b8d8\") "
Jul 24 14:51:56.850137 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:56.850112    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca074a3-3b95-4cc9-991b-41b03879b8d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ca074a3-3b95-4cc9-991b-41b03879b8d8" (UID: "4ca074a3-3b95-4cc9-991b-41b03879b8d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:51:56.950825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:56.950746    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca074a3-3b95-4cc9-991b-41b03879b8d8-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:51:57.666352 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:57.666328    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"
Jul 24 14:51:57.671273 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:57.671247    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm" event={"ID":"4ca074a3-3b95-4cc9-991b-41b03879b8d8","Type":"ContainerDied","Data":"876bd4a0de0fce439b13672e4a2bdd087dd896a0650112d3b311039c601f1a15"}
Jul 24 14:51:57.671391 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:51:57.671286    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="876bd4a0de0fce439b13672e4a2bdd087dd896a0650112d3b311039c601f1a15"
Jul 24 14:52:07.257780 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:07.257744    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:52:08.798765 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:08.798737    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"]
Jul 24 14:52:08.800524 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:08.800502    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2e2d5bdd-9fwlm"]
Jul 24 14:52:08.962170 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:08.962143    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"]
Jul 24 14:52:08.962355 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:08.962333    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy" containerID="cri-o://4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13" gracePeriod=30
Jul 24 14:52:09.670379 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:09.670346    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca074a3-3b95-4cc9-991b-41b03879b8d8" path="/var/lib/kubelet/pods/4ca074a3-3b95-4cc9-991b-41b03879b8d8/volumes"
Jul 24 14:52:10.278034 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.278000    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"]
Jul 24 14:52:10.278412 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.278399    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca074a3-3b95-4cc9-991b-41b03879b8d8" containerName="revoke-keys"
Jul 24 14:52:10.278466 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.278414    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca074a3-3b95-4cc9-991b-41b03879b8d8" containerName="revoke-keys"
Jul 24 14:52:10.278501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.278493    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca074a3-3b95-4cc9-991b-41b03879b8d8" containerName="revoke-keys"
Jul 24 14:52:10.281800 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.281781    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.285242 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.285211    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-9671202b-gw-openshift-default-dockercfg-sq4kt\""
Jul 24 14:52:10.318226 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.318201    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"]
Jul 24 14:52:10.362675 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362636    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-credential-socket\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.362853 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362706    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6502c64-affc-428f-9af6-cf1da7464e01-istiod-ca-cert\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.362853 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362782    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-data\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.362853 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362805    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6502c64-affc-428f-9af6-cf1da7464e01-istio-podinfo\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.363051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362882    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-istio-token\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.363051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362914    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-envoy\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.363051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.362975    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-certs\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.363051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.363003    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-socket\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.363233 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.363092    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54rdx\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-kube-api-access-54rdx\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.463911 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.463870    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6502c64-affc-428f-9af6-cf1da7464e01-istiod-ca-cert\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.463936    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-data\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.463959    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6502c64-affc-428f-9af6-cf1da7464e01-istio-podinfo\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.463983    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-istio-token\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464003    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-envoy\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464035    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-certs\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464062    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-socket\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464378 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464101    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54rdx\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-kube-api-access-54rdx\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464378 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464166    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-credential-socket\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464465 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464418    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-data\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464556 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464532    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-certs\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464621 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464590    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-socket\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464735 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464720    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-credential-socket\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.464771 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.464755    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6502c64-affc-428f-9af6-cf1da7464e01-istiod-ca-cert\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.466306 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.466287    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-envoy\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.466610 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.466595    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6502c64-affc-428f-9af6-cf1da7464e01-istio-podinfo\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.473861 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.473808    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-istio-token\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.475077 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.475052    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54rdx\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-kube-api-access-54rdx\") pod \"e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") " pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.591732 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.591653    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:10.729782 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.729762    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"]
Jul 24 14:52:10.732428 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:52:10.732399    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6502c64_affc_428f_9af6_cf1da7464e01.slice/crio-a4931e609ec1818d3b212259e5322b1e7749e040dc3bae09399466df422c9f44 WatchSource:0}: Error finding container a4931e609ec1818d3b212259e5322b1e7749e040dc3bae09399466df422c9f44: Status 404 returned error can't find the container with id a4931e609ec1818d3b212259e5322b1e7749e040dc3bae09399466df422c9f44
Jul 24 14:52:10.734688 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.734650    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:10.734781 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.734733    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:10.734819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:10.734780    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:11.747195 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:11.747155    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" event={"ID":"e6502c64-affc-428f-9af6-cf1da7464e01","Type":"ContainerStarted","Data":"ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"}
Jul 24 14:52:11.747588 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:11.747246    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" event={"ID":"e6502c64-affc-428f-9af6-cf1da7464e01","Type":"ContainerStarted","Data":"a4931e609ec1818d3b212259e5322b1e7749e040dc3bae09399466df422c9f44"}
Jul 24 14:52:11.771463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:11.771412    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" podStartSLOduration=1.771394281 podStartE2EDuration="1.771394281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:52:11.771088018 +0000 UTC m=+1788.555788481" watchObservedRunningTime="2026-07-24 14:52:11.771394281 +0000 UTC m=+1788.556094747"
Jul 24 14:52:12.592663 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:12.592625    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:12.597684 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:12.597664    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:12.752009 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:12.751983    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:12.752857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:12.752818    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:13.529004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:13.528963    2584 patch_prober.go:28] interesting pod/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 14:52:13.529165 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:13.529026    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:52:14.226720 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.226666    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:52:14.298815 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.298789    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-podinfo\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.298815 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.298817    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-socket\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299022 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.298968    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-certs\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299065 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299031    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdnjp\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-kube-api-access-xdnjp\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299105 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299073    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:14.299105 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299090    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-token\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299201 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299132    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-credential-socket\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299201 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299164    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-envoy\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299274 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299203    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-data\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299274 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299228    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istiod-ca-cert\") pod \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\" (UID: \"4bded7fa-3eda-447a-a56f-d5b0eb96aa92\") "
Jul 24 14:52:14.299274 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299233    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:14.299489 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299464    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:14.299629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299587    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.299629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299625    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.299744 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299640    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.299744 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.299699    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:52:14.300152 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.300122    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-data" (OuterVolumeSpecName: "istio-data") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:14.301705 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.301642    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-token" (OuterVolumeSpecName: "istio-token") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:52:14.301829 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.301768    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 14:52:14.302030 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.302007    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-kube-api-access-xdnjp" (OuterVolumeSpecName: "kube-api-access-xdnjp") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "kube-api-access-xdnjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:52:14.302132 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.302031    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4bded7fa-3eda-447a-a56f-d5b0eb96aa92" (UID: "4bded7fa-3eda-447a-a56f-d5b0eb96aa92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:14.400938 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.400913    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdnjp\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-kube-api-access-xdnjp\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.400938 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.400936    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.400938 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.400945    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.401147 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.400952    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.401147 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.400960    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.401147 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.400967    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4bded7fa-3eda-447a-a56f-d5b0eb96aa92-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:14.767249 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.767213    2584 generic.go:356] "Generic (PLEG): container finished" podID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerID="4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13" exitCode=0
Jul 24 14:52:14.767459 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.767282    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"
Jul 24 14:52:14.767459 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.767309    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" event={"ID":"4bded7fa-3eda-447a-a56f-d5b0eb96aa92","Type":"ContainerDied","Data":"4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13"}
Jul 24 14:52:14.767459 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.767374    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs" event={"ID":"4bded7fa-3eda-447a-a56f-d5b0eb96aa92","Type":"ContainerDied","Data":"dd9f9ab3bdb8b7bfdeeb70651cdc016119cb967b9c2286081d33888eaa239cc5"}
Jul 24 14:52:14.767459 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.767435    2584 scope.go:117] "RemoveContainer" containerID="4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13"
Jul 24 14:52:14.779004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.778987    2584 scope.go:117] "RemoveContainer" containerID="bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"
Jul 24 14:52:14.787337 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.787320    2584 scope.go:117] "RemoveContainer" containerID="4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13"
Jul 24 14:52:14.787597 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:52:14.787575    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13\": container with ID starting with 4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13 not found: ID does not exist" containerID="4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13"
Jul 24 14:52:14.787670 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.787605    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13"} err="failed to get container status \"4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13\": rpc error: code = NotFound desc = could not find container \"4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13\": container with ID starting with 4d6cb605d254b12876fd06dc674a55fca24eba61fc5ecd17f39bbe20983f0b13 not found: ID does not exist"
Jul 24 14:52:14.787670 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.787622    2584 scope.go:117] "RemoveContainer" containerID="bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"
Jul 24 14:52:14.787906 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:52:14.787878    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458\": container with ID starting with bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458 not found: ID does not exist" containerID="bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"
Jul 24 14:52:14.787991 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.787916    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458"} err="failed to get container status \"bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458\": rpc error: code = NotFound desc = could not find container \"bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458\": container with ID starting with bac6a6b6cbafa2a934a4a53d55a63192356c230b7957339647a7123b7d12a458 not found: ID does not exist"
Jul 24 14:52:14.803538 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.803512    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"]
Jul 24 14:52:14.813965 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:14.813945    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2e2d5bdd-openshift-default-b9675c47b-rjsbs"]
Jul 24 14:52:15.671604 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:15.671528    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" path="/var/lib/kubelet/pods/4bded7fa-3eda-447a-a56f-d5b0eb96aa92/volumes"
Jul 24 14:52:18.462300 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:18.462269    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:52:23.680164 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:23.680138    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:52:23.681820 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:23.681800    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:52:26.845047 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:26.845011    2584 generic.go:356] "Generic (PLEG): container finished" podID="e6502c64-affc-428f-9af6-cf1da7464e01" containerID="ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983" exitCode=0
Jul 24 14:52:26.845546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:26.845082    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" event={"ID":"e6502c64-affc-428f-9af6-cf1da7464e01","Type":"ContainerDied","Data":"ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"}
Jul 24 14:52:26.845546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:26.845426    2584 scope.go:117] "RemoveContainer" containerID="ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"
Jul 24 14:52:26.846330 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:26.846304    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:26.846417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:26.846363    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:26.846417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:26.846392    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:27.751966 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:27.751923    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:27.851528 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:27.851493    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" event={"ID":"e6502c64-affc-428f-9af6-cf1da7464e01","Type":"ContainerStarted","Data":"4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005"}
Jul 24 14:52:28.593959 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:28.593926    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:28.594307 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:28.594282    2584 patch_prober.go:28] interesting pod/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 14:52:28.594472 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:28.594446    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 14:52:29.593243 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:29.592782    2584 patch_prober.go:28] interesting pod/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 14:52:29.593243 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:29.592866    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 14:52:30.592617 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:30.592571    2584 patch_prober.go:28] interesting pod/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 14:52:30.592823 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:30.592640    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 14:52:31.596244 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:31.596160    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:31.596626 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:31.596419    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:31.597142 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:31.597127    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:38.433421 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.433388    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"]
Jul 24 14:52:38.433881 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.433862    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy"
Jul 24 14:52:38.433955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.433883    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy"
Jul 24 14:52:38.433955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.433918    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy"
Jul 24 14:52:38.433955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.433927    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy"
Jul 24 14:52:38.434050 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.434016    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy"
Jul 24 14:52:38.437495 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.437479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:38.440348 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.440333    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:52:38.440418 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.440388    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:52:38.449288 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.449269    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"]
Jul 24 14:52:38.508938 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.508908    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dff203f-87a1-4ac7-88ad-23bd963e4e6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9671202b-44c55\" (UID: \"2dff203f-87a1-4ac7-88ad-23bd963e4e6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:38.609722 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.609691    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dff203f-87a1-4ac7-88ad-23bd963e4e6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9671202b-44c55\" (UID: \"2dff203f-87a1-4ac7-88ad-23bd963e4e6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:38.610289 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.610271    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dff203f-87a1-4ac7-88ad-23bd963e4e6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9671202b-44c55\" (UID: \"2dff203f-87a1-4ac7-88ad-23bd963e4e6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:38.747410 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.747342    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:38.901577 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:38.901553    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"]
Jul 24 14:52:38.904034 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:52:38.904009    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dff203f_87a1_4ac7_88ad_23bd963e4e6a.slice/crio-73c448ce07dfa5746e801c8df45895c074fdc7fdf7642cc4482ceac31a1aa30a WatchSource:0}: Error finding container 73c448ce07dfa5746e801c8df45895c074fdc7fdf7642cc4482ceac31a1aa30a: Status 404 returned error can't find the container with id 73c448ce07dfa5746e801c8df45895c074fdc7fdf7642cc4482ceac31a1aa30a
Jul 24 14:52:39.913069 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:39.913036    2584 generic.go:356] "Generic (PLEG): container finished" podID="2dff203f-87a1-4ac7-88ad-23bd963e4e6a" containerID="f4c871239531652eafc71d6d75e319ce844738bc8a6c308866196b6f3fb4110c" exitCode=0
Jul 24 14:52:39.913523 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:39.913112    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55" event={"ID":"2dff203f-87a1-4ac7-88ad-23bd963e4e6a","Type":"ContainerDied","Data":"f4c871239531652eafc71d6d75e319ce844738bc8a6c308866196b6f3fb4110c"}
Jul 24 14:52:39.913523 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:39.913165    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55" event={"ID":"2dff203f-87a1-4ac7-88ad-23bd963e4e6a","Type":"ContainerStarted","Data":"73c448ce07dfa5746e801c8df45895c074fdc7fdf7642cc4482ceac31a1aa30a"}
Jul 24 14:52:41.046296 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.046267    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:41.131970 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.131931    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dff203f-87a1-4ac7-88ad-23bd963e4e6a-maas-api-service-ca\") pod \"2dff203f-87a1-4ac7-88ad-23bd963e4e6a\" (UID: \"2dff203f-87a1-4ac7-88ad-23bd963e4e6a\") "
Jul 24 14:52:41.132260 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.132239    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dff203f-87a1-4ac7-88ad-23bd963e4e6a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2dff203f-87a1-4ac7-88ad-23bd963e4e6a" (UID: "2dff203f-87a1-4ac7-88ad-23bd963e4e6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:52:41.233362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.233294    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dff203f-87a1-4ac7-88ad-23bd963e4e6a-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:41.924383 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.924358    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"
Jul 24 14:52:41.924526 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.924395    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55" event={"ID":"2dff203f-87a1-4ac7-88ad-23bd963e4e6a","Type":"ContainerDied","Data":"73c448ce07dfa5746e801c8df45895c074fdc7fdf7642cc4482ceac31a1aa30a"}
Jul 24 14:52:41.924526 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:41.924439    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73c448ce07dfa5746e801c8df45895c074fdc7fdf7642cc4482ceac31a1aa30a"
Jul 24 14:52:52.065805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:52.065772    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:52:53.535309 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:53.535275    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"]
Jul 24 14:52:53.538938 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:53.538912    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9671202b-44c55"]
Jul 24 14:52:53.671957 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:53.671889    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dff203f-87a1-4ac7-88ad-23bd963e4e6a" path="/var/lib/kubelet/pods/2dff203f-87a1-4ac7-88ad-23bd963e4e6a/volumes"
Jul 24 14:52:53.723697 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:53.723666    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"]
Jul 24 14:52:53.723930 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:53.723885    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy" containerID="cri-o://4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005" gracePeriod=30
Jul 24 14:52:54.553957 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.553925    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"]
Jul 24 14:52:54.554312 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.554303    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dff203f-87a1-4ac7-88ad-23bd963e4e6a" containerName="revoke-keys"
Jul 24 14:52:54.554358 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.554314    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dff203f-87a1-4ac7-88ad-23bd963e4e6a" containerName="revoke-keys"
Jul 24 14:52:54.554392 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.554381    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dff203f-87a1-4ac7-88ad-23bd963e4e6a" containerName="revoke-keys"
Jul 24 14:52:54.554428 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.554392    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bded7fa-3eda-447a-a56f-d5b0eb96aa92" containerName="istio-proxy"
Jul 24 14:52:54.558599 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.558582    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.561251 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.561231    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6b842b2f-openshift-default-dockercfg-kj6st\""
Jul 24 14:52:54.571537 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.571516    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"]
Jul 24 14:52:54.650570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650540    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-envoy\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650758 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-socket\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650758 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650612    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-podinfo\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650758 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650720    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-certs\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650761    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scw5c\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-kube-api-access-scw5c\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650867    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-token\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650932    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-data\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650958    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-credential-socket\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.650999 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.650982    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istiod-ca-cert\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752214 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752177    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-podinfo\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752226    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-certs\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752248    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scw5c\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-kube-api-access-scw5c\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752274    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-token\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752306    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-data\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-credential-socket\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752345    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istiod-ca-cert\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752474    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-envoy\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-socket\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752628    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-certs\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752646    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-data\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752850 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752744    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-credential-socket\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.752920 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.752899    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-socket\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.753291 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.753268    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istiod-ca-cert\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.754721 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.754696    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-envoy\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.755025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.755008    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-podinfo\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.761763 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.761741    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scw5c\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-kube-api-access-scw5c\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.761896 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.761876    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-token\") pod \"e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") " pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:54.868735 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:54.868674    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:55.009808 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:55.009781    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"]
Jul 24 14:52:55.011760 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:52:55.011734    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78077318_1fbc_4090_bc5b_8dc2c5db2bb4.slice/crio-e6e309acc874bad91f979f3bb1c91a479168838c91cd9f1e454db23d8ccb7a1c WatchSource:0}: Error finding container e6e309acc874bad91f979f3bb1c91a479168838c91cd9f1e454db23d8ccb7a1c: Status 404 returned error can't find the container with id e6e309acc874bad91f979f3bb1c91a479168838c91cd9f1e454db23d8ccb7a1c
Jul 24 14:52:55.013878 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:55.013820    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:55.013965 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:55.013908    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:55.013965 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:55.013944    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:52:55.997097 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:55.997057    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" event={"ID":"78077318-1fbc-4090-bc5b-8dc2c5db2bb4","Type":"ContainerStarted","Data":"cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"}
Jul 24 14:52:55.997523 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:55.997114    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" event={"ID":"78077318-1fbc-4090-bc5b-8dc2c5db2bb4","Type":"ContainerStarted","Data":"e6e309acc874bad91f979f3bb1c91a479168838c91cd9f1e454db23d8ccb7a1c"}
Jul 24 14:52:56.024521 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:56.024476    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podStartSLOduration=2.024462706 podStartE2EDuration="2.024462706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:52:56.022725517 +0000 UTC m=+1832.807425990" watchObservedRunningTime="2026-07-24 14:52:56.024462706 +0000 UTC m=+1832.809163169"
Jul 24 14:52:56.869022 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:56.868987    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:56.873673 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:56.873652    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:57.003606 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:57.003570    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:57.004621 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:57.004597    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:52:58.985045 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:58.984986    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:59.015908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.015868    2584 generic.go:356] "Generic (PLEG): container finished" podID="e6502c64-affc-428f-9af6-cf1da7464e01" containerID="4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005" exitCode=0
Jul 24 14:52:59.015908 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.015906    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"
Jul 24 14:52:59.016086 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.015930    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" event={"ID":"e6502c64-affc-428f-9af6-cf1da7464e01","Type":"ContainerDied","Data":"4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005"}
Jul 24 14:52:59.016086 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.015983    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx" event={"ID":"e6502c64-affc-428f-9af6-cf1da7464e01","Type":"ContainerDied","Data":"a4931e609ec1818d3b212259e5322b1e7749e040dc3bae09399466df422c9f44"}
Jul 24 14:52:59.016086 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.016006    2584 scope.go:117] "RemoveContainer" containerID="4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005"
Jul 24 14:52:59.026707 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.026691    2584 scope.go:117] "RemoveContainer" containerID="ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"
Jul 24 14:52:59.035907 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.035887    2584 scope.go:117] "RemoveContainer" containerID="4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005"
Jul 24 14:52:59.036150 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:52:59.036134    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005\": container with ID starting with 4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005 not found: ID does not exist" containerID="4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005"
Jul 24 14:52:59.036196 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.036159    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005"} err="failed to get container status \"4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005\": rpc error: code = NotFound desc = could not find container \"4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005\": container with ID starting with 4998240d63764ba924d5f7caf817e98c7c22153685611c9bdc269ede7858b005 not found: ID does not exist"
Jul 24 14:52:59.036196 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.036176    2584 scope.go:117] "RemoveContainer" containerID="ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"
Jul 24 14:52:59.036447 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:52:59.036431    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983\": container with ID starting with ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983 not found: ID does not exist" containerID="ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"
Jul 24 14:52:59.036491 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.036453    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983"} err="failed to get container status \"ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983\": rpc error: code = NotFound desc = could not find container \"ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983\": container with ID starting with ca96e1cbdfa39dba2fe5479e6e05cd070bb587858b7660b020a5419cbefab983 not found: ID does not exist"
Jul 24 14:52:59.092945 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.092919    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-certs\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.092970    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54rdx\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-kube-api-access-54rdx\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093057 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093047    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6502c64-affc-428f-9af6-cf1da7464e01-istiod-ca-cert\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093068    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-data\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093113    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-istio-token\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093173    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:59.093312 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093236    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-envoy\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093361 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093321    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-credential-socket\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093361 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093354    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6502c64-affc-428f-9af6-cf1da7464e01-istio-podinfo\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093465 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093379    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-socket\") pod \"e6502c64-affc-428f-9af6-cf1da7464e01\" (UID: \"e6502c64-affc-428f-9af6-cf1da7464e01\") "
Jul 24 14:52:59.093465 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093450    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6502c64-affc-428f-9af6-cf1da7464e01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:52:59.093570 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093550    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:59.093805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093787    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.093917 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093809    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.093979 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.093938    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:59.094098 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.094069    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:59.095370 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.095343    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:52:59.095690 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.095670    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-kube-api-access-54rdx" (OuterVolumeSpecName: "kube-api-access-54rdx") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "kube-api-access-54rdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:52:59.095939 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.095920    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6502c64-affc-428f-9af6-cf1da7464e01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 14:52:59.095939 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.095927    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6502c64-affc-428f-9af6-cf1da7464e01" (UID: "e6502c64-affc-428f-9af6-cf1da7464e01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:52:59.194253 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194221    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54rdx\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-kube-api-access-54rdx\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.194253 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194248    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6502c64-affc-428f-9af6-cf1da7464e01-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.194253 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194257    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.194463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194267    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6502c64-affc-428f-9af6-cf1da7464e01-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.194463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194275    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.194463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194282    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6502c64-affc-428f-9af6-cf1da7464e01-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.194463 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.194289    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6502c64-affc-428f-9af6-cf1da7464e01-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:52:59.348452 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.348419    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"]
Jul 24 14:52:59.355991 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.355967    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-9671202b-gw-openshift-default-857845b4fb-gw2fx"]
Jul 24 14:52:59.670803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:52:59.670725    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" path="/var/lib/kubelet/pods/e6502c64-affc-428f-9af6-cf1da7464e01/volumes"
Jul 24 14:53:01.677365 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:01.677333    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:53:09.071630 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:09.071595    2584 generic.go:356] "Generic (PLEG): container finished" podID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerID="cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14" exitCode=0
Jul 24 14:53:09.072035 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:09.071666    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" event={"ID":"78077318-1fbc-4090-bc5b-8dc2c5db2bb4","Type":"ContainerDied","Data":"cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"}
Jul 24 14:53:09.072121 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:09.072102    2584 scope.go:117] "RemoveContainer" containerID="cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"
Jul 24 14:53:09.073159 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:09.073134    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:09.073278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:09.073195    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:09.073278 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:09.073224    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:10.078787 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:10.078757    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" event={"ID":"78077318-1fbc-4090-bc5b-8dc2c5db2bb4","Type":"ContainerStarted","Data":"206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01"}
Jul 24 14:53:10.869687 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:10.869656    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:53:10.870893 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:10.870863    2584 patch_prober.go:28] interesting pod/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 14:53:10.871026 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:10.870921    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 14:53:11.869491 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:11.869458    2584 patch_prober.go:28] interesting pod/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 14:53:11.869982 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:11.869518    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 14:53:12.003722 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:12.003680    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:53:12.869609 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:12.869573    2584 patch_prober.go:28] interesting pod/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 14:53:12.870055 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:12.869636    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 14:53:13.869571 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:13.869537    2584 patch_prober.go:28] interesting pod/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 24 14:53:13.869751 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:13.869598    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 24 14:53:14.872983 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:14.872954    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:53:14.873919 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:14.873901    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:53:23.449522 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.449490    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"]
Jul 24 14:53:23.450036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.449880    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy"
Jul 24 14:53:23.450036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.449892    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy"
Jul 24 14:53:23.450036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.449900    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy"
Jul 24 14:53:23.450036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.449905    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy"
Jul 24 14:53:23.450036 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.449962    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy"
Jul 24 14:53:23.453221 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.453203    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:23.456077 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.456055    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:53:23.456198 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.456059    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:53:23.461762 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.461741    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"]
Jul 24 14:53:23.512691 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.512670    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2304ea69-2a66-458b-a4a4-2a9b256b496c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg\" (UID: \"2304ea69-2a66-458b-a4a4-2a9b256b496c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:23.613045 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.613022    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2304ea69-2a66-458b-a4a4-2a9b256b496c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg\" (UID: \"2304ea69-2a66-458b-a4a4-2a9b256b496c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:23.616874 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.616858    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:53:23.624371 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.624348    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2304ea69-2a66-458b-a4a4-2a9b256b496c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg\" (UID: \"2304ea69-2a66-458b-a4a4-2a9b256b496c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:23.768392 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.768370    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:53:23.775564 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.775546    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:23.908478 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:23.908455    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"]
Jul 24 14:53:23.910267 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:53:23.910239    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2304ea69_2a66_458b_a4a4_2a9b256b496c.slice/crio-51cee04fd988eb3f16409e5d89bb4a89c1bc184f123ba3565737ba51ae6e08b0 WatchSource:0}: Error finding container 51cee04fd988eb3f16409e5d89bb4a89c1bc184f123ba3565737ba51ae6e08b0: Status 404 returned error can't find the container with id 51cee04fd988eb3f16409e5d89bb4a89c1bc184f123ba3565737ba51ae6e08b0
Jul 24 14:53:24.154877 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:24.154825    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg" event={"ID":"2304ea69-2a66-458b-a4a4-2a9b256b496c","Type":"ContainerStarted","Data":"51cee04fd988eb3f16409e5d89bb4a89c1bc184f123ba3565737ba51ae6e08b0"}
Jul 24 14:53:25.162354 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:25.162322    2584 generic.go:356] "Generic (PLEG): container finished" podID="2304ea69-2a66-458b-a4a4-2a9b256b496c" containerID="f8a625b8e33211a86ebb309c547c67020c6dd1d6103699567138826e1ff955a7" exitCode=0
Jul 24 14:53:25.162730 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:25.162418    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg" event={"ID":"2304ea69-2a66-458b-a4a4-2a9b256b496c","Type":"ContainerDied","Data":"f8a625b8e33211a86ebb309c547c67020c6dd1d6103699567138826e1ff955a7"}
Jul 24 14:53:26.300712 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:26.300693    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:26.335393 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:26.335370    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2304ea69-2a66-458b-a4a4-2a9b256b496c-maas-api-service-ca\") pod \"2304ea69-2a66-458b-a4a4-2a9b256b496c\" (UID: \"2304ea69-2a66-458b-a4a4-2a9b256b496c\") "
Jul 24 14:53:26.335716 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:26.335698    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2304ea69-2a66-458b-a4a4-2a9b256b496c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2304ea69-2a66-458b-a4a4-2a9b256b496c" (UID: "2304ea69-2a66-458b-a4a4-2a9b256b496c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:53:26.436139 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:26.436075    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2304ea69-2a66-458b-a4a4-2a9b256b496c-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:27.174874 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:27.174844    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"
Jul 24 14:53:27.175052 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:27.174868    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg" event={"ID":"2304ea69-2a66-458b-a4a4-2a9b256b496c","Type":"ContainerDied","Data":"51cee04fd988eb3f16409e5d89bb4a89c1bc184f123ba3565737ba51ae6e08b0"}
Jul 24 14:53:27.175052 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:27.174947    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51cee04fd988eb3f16409e5d89bb4a89c1bc184f123ba3565737ba51ae6e08b0"
Jul 24 14:53:36.860684 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:36.860647    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:53:38.629927 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:38.629892    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"]
Jul 24 14:53:38.633653 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:38.633630    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6b842b2f-pjhdg"]
Jul 24 14:53:39.670265 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:39.670230    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2304ea69-2a66-458b-a4a4-2a9b256b496c" path="/var/lib/kubelet/pods/2304ea69-2a66-458b-a4a4-2a9b256b496c/volumes"
Jul 24 14:53:39.991389 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:39.991309    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"]
Jul 24 14:53:39.991580 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:39.991556    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy" containerID="cri-o://206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01" gracePeriod=30
Jul 24 14:53:40.829006 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.828964    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"]
Jul 24 14:53:40.829559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.829542    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2304ea69-2a66-458b-a4a4-2a9b256b496c" containerName="revoke-keys"
Jul 24 14:53:40.829559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.829560    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2304ea69-2a66-458b-a4a4-2a9b256b496c" containerName="revoke-keys"
Jul 24 14:53:40.829732 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.829652    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2304ea69-2a66-458b-a4a4-2a9b256b496c" containerName="revoke-keys"
Jul 24 14:53:40.829732 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.829666    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6502c64-affc-428f-9af6-cf1da7464e01" containerName="istio-proxy"
Jul 24 14:53:40.833344 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.833321    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.836423 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.836400    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ab2fb2a0-openshift-default-dockercfg-txm2k\""
Jul 24 14:53:40.847211 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.847186    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"]
Jul 24 14:53:40.849317 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849298    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-socket\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849464 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849324    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-podinfo\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849464 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849351    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-data\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849464 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849378    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-envoy\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849464 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849447    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istiod-ca-cert\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849673 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849474    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnl57\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-kube-api-access-dnl57\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849673 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849513    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-certs\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849673 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849546    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-token\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.849673 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.849615    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-credential-socket\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950434 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950397    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-credential-socket\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950434 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-socket\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950685 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950454    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-podinfo\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950685 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-data\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950685 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950656    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-envoy\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950854 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950683    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istiod-ca-cert\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950854 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950709    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnl57\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-kube-api-access-dnl57\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950854 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950757    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-certs\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950854 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950798    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-token\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.950854 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950819    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-socket\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.951114 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.950897    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-credential-socket\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.951114 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.951016    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-data\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.951219 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.951183    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-certs\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.951594 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.951573    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istiod-ca-cert\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.952944 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.952916    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-envoy\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.953203 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.953185    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-podinfo\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.959821 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.959795    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-token\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:40.960256 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:40.960239    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnl57\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-kube-api-access-dnl57\") pod \"e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") " pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:41.144068 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:41.143981    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:41.292502 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:41.292475    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"]
Jul 24 14:53:41.294448 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:53:41.294424    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0608695b_fb0c_4341_8fd0_91a6ec6d0b87.slice/crio-8c6fa8aa4d2baf5dc5ca1f311d0bd86b0c35ae969c2e9a6b88e55d532c269da8 WatchSource:0}: Error finding container 8c6fa8aa4d2baf5dc5ca1f311d0bd86b0c35ae969c2e9a6b88e55d532c269da8: Status 404 returned error can't find the container with id 8c6fa8aa4d2baf5dc5ca1f311d0bd86b0c35ae969c2e9a6b88e55d532c269da8
Jul 24 14:53:41.296884 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:41.296823    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:41.296997 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:41.296917    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:41.296997 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:41.296944    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:42.249380 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:42.249348    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" event={"ID":"0608695b-fb0c-4341-8fd0-91a6ec6d0b87","Type":"ContainerStarted","Data":"e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"}
Jul 24 14:53:42.249733 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:42.249400    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" event={"ID":"0608695b-fb0c-4341-8fd0-91a6ec6d0b87","Type":"ContainerStarted","Data":"8c6fa8aa4d2baf5dc5ca1f311d0bd86b0c35ae969c2e9a6b88e55d532c269da8"}
Jul 24 14:53:42.275401 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:42.275358    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" podStartSLOduration=2.275341382 podStartE2EDuration="2.275341382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:53:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:53:42.275258234 +0000 UTC m=+1879.059958699" watchObservedRunningTime="2026-07-24 14:53:42.275341382 +0000 UTC m=+1879.060041890"
Jul 24 14:53:43.144359 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:43.144329    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:43.148675 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:43.148652    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:43.254716 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:43.254686    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:43.255623 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:43.255604    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:44.875476 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:44.875444    2584 patch_prober.go:28] interesting pod/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 24 14:53:44.875917 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:44.875502    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:53:45.241586 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.241563    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:53:45.266731 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.266706    2584 generic.go:356] "Generic (PLEG): container finished" podID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerID="206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01" exitCode=0
Jul 24 14:53:45.266868 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.266748    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"
Jul 24 14:53:45.266868 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.266784    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" event={"ID":"78077318-1fbc-4090-bc5b-8dc2c5db2bb4","Type":"ContainerDied","Data":"206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01"}
Jul 24 14:53:45.266954 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.266888    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh" event={"ID":"78077318-1fbc-4090-bc5b-8dc2c5db2bb4","Type":"ContainerDied","Data":"e6e309acc874bad91f979f3bb1c91a479168838c91cd9f1e454db23d8ccb7a1c"}
Jul 24 14:53:45.266954 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.266918    2584 scope.go:117] "RemoveContainer" containerID="206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01"
Jul 24 14:53:45.275798 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.275780    2584 scope.go:117] "RemoveContainer" containerID="cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"
Jul 24 14:53:45.283625 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.283609    2584 scope.go:117] "RemoveContainer" containerID="206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01"
Jul 24 14:53:45.283876 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:53:45.283855    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01\": container with ID starting with 206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01 not found: ID does not exist" containerID="206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01"
Jul 24 14:53:45.283922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.283888    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01"} err="failed to get container status \"206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01\": rpc error: code = NotFound desc = could not find container \"206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01\": container with ID starting with 206c1f3ca80a30791d87257201bea0af47be74d0abdb98d3f9f8bd17d0283e01 not found: ID does not exist"
Jul 24 14:53:45.283922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.283914    2584 scope.go:117] "RemoveContainer" containerID="cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"
Jul 24 14:53:45.284190 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:53:45.284168    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14\": container with ID starting with cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14 not found: ID does not exist" containerID="cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"
Jul 24 14:53:45.284245 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.284200    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14"} err="failed to get container status \"cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14\": rpc error: code = NotFound desc = could not find container \"cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14\": container with ID starting with cb84246478e69993a169a657d87527261e442297651c797183688845c222cc14 not found: ID does not exist"
Jul 24 14:53:45.293486 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293468    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-token\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293556 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293499    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-socket\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293556 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293519    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-credential-socket\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293561    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-data\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293635 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293607    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-certs\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293730 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293639    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-podinfo\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293730 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293722    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scw5c\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-kube-api-access-scw5c\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293858 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293748    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istiod-ca-cert\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293858 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293773    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:53:45.293858 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293776    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-envoy\") pod \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\" (UID: \"78077318-1fbc-4090-bc5b-8dc2c5db2bb4\") "
Jul 24 14:53:45.293858 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.293814    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:53:45.294546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.294193    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.294546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.294216    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.294546 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.294323    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:53:45.294740 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.294584    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-data" (OuterVolumeSpecName: "istio-data") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:53:45.294892 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.294820    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:53:45.295728 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.295707    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-token" (OuterVolumeSpecName: "istio-token") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:53:45.296056 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.296035    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-kube-api-access-scw5c" (OuterVolumeSpecName: "kube-api-access-scw5c") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "kube-api-access-scw5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:53:45.296248 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.296226    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 14:53:45.296303 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.296235    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78077318-1fbc-4090-bc5b-8dc2c5db2bb4" (UID: "78077318-1fbc-4090-bc5b-8dc2c5db2bb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:53:45.395583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395512    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.395583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395541    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.395583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395553    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scw5c\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-kube-api-access-scw5c\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.395583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395565    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.395583 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395577    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.395824 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395590    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.395824 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.395600    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78077318-1fbc-4090-bc5b-8dc2c5db2bb4-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:53:45.596852 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.596805    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"]
Jul 24 14:53:45.598710 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.598688    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6b842b2f-openshift-default-77597d49bd-82rzh"]
Jul 24 14:53:45.670903 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:45.670876    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" path="/var/lib/kubelet/pods/78077318-1fbc-4090-bc5b-8dc2c5db2bb4/volumes"
Jul 24 14:53:48.490268 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:48.490231    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:53:56.333759 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:56.333685    2584 generic.go:356] "Generic (PLEG): container finished" podID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerID="e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e" exitCode=0
Jul 24 14:53:56.333759 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:56.333729    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" event={"ID":"0608695b-fb0c-4341-8fd0-91a6ec6d0b87","Type":"ContainerDied","Data":"e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"}
Jul 24 14:53:56.334196 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:56.334104    2584 scope.go:117] "RemoveContainer" containerID="e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"
Jul 24 14:53:56.337417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:56.335355    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:56.337417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:56.335441    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:56.337417 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:56.335480    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:53:57.340627 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:57.340597    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" event={"ID":"0608695b-fb0c-4341-8fd0-91a6ec6d0b87","Type":"ContainerStarted","Data":"abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53"}
Jul 24 14:53:58.144366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:58.144333    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:58.145282 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:58.145259    2584 patch_prober.go:28] interesting pod/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 24 14:53:58.145358 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:58.145318    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 24 14:53:58.255107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:58.255079    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:53:59.144968 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:59.144932    2584 patch_prober.go:28] interesting pod/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 24 14:53:59.145403 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:53:59.145003    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 24 14:54:00.144946 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:00.144916    2584 patch_prober.go:28] interesting pod/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 24 14:54:00.145132 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:00.144964    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 24 14:54:01.148823 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:01.148792    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:54:01.149666 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:01.149650    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:54:13.798213 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.798178    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"]
Jul 24 14:54:13.798733 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.798714    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy"
Jul 24 14:54:13.798810 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.798737    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy"
Jul 24 14:54:13.798810 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.798766    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy"
Jul 24 14:54:13.798810 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.798775    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy"
Jul 24 14:54:13.799012 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.798930    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy"
Jul 24 14:54:13.802259 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.802240    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:13.806557 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.806539    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:54:13.806885 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.806869    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:54:13.813960 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.813939    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"]
Jul 24 14:54:13.952967 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:13.952934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed29299-dfea-41d5-a315-5146c26a852b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85\" (UID: \"8ed29299-dfea-41d5-a315-5146c26a852b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:14.053762 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:14.053682    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed29299-dfea-41d5-a315-5146c26a852b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85\" (UID: \"8ed29299-dfea-41d5-a315-5146c26a852b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:14.054250 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:14.054233    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed29299-dfea-41d5-a315-5146c26a852b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85\" (UID: \"8ed29299-dfea-41d5-a315-5146c26a852b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:14.112364 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:14.112339    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:14.455284 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:14.455258    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"]
Jul 24 14:54:14.456706 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:54:14.456684    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed29299_dfea_41d5_a315_5146c26a852b.slice/crio-d008a8dc44a0338e33714de80796ba39ee2aa7a3c6c7e9b087446aa497961b3a WatchSource:0}: Error finding container d008a8dc44a0338e33714de80796ba39ee2aa7a3c6c7e9b087446aa497961b3a: Status 404 returned error can't find the container with id d008a8dc44a0338e33714de80796ba39ee2aa7a3c6c7e9b087446aa497961b3a
Jul 24 14:54:15.432680 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:15.432650    2584 generic.go:356] "Generic (PLEG): container finished" podID="8ed29299-dfea-41d5-a315-5146c26a852b" containerID="5c80f44d8108400b9914fb7ec140f63cd4a8dd1e93c437c426718b9c99e1bf62" exitCode=0
Jul 24 14:54:15.433120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:15.432738    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85" event={"ID":"8ed29299-dfea-41d5-a315-5146c26a852b","Type":"ContainerDied","Data":"5c80f44d8108400b9914fb7ec140f63cd4a8dd1e93c437c426718b9c99e1bf62"}
Jul 24 14:54:15.433120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:15.432803    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85" event={"ID":"8ed29299-dfea-41d5-a315-5146c26a852b","Type":"ContainerStarted","Data":"d008a8dc44a0338e33714de80796ba39ee2aa7a3c6c7e9b087446aa497961b3a"}
Jul 24 14:54:16.565762 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:16.565738    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:16.676056 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:16.676028    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed29299-dfea-41d5-a315-5146c26a852b-maas-api-service-ca\") pod \"8ed29299-dfea-41d5-a315-5146c26a852b\" (UID: \"8ed29299-dfea-41d5-a315-5146c26a852b\") "
Jul 24 14:54:16.676362 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:16.676341    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ed29299-dfea-41d5-a315-5146c26a852b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ed29299-dfea-41d5-a315-5146c26a852b" (UID: "8ed29299-dfea-41d5-a315-5146c26a852b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:54:16.777372 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:16.777346    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ed29299-dfea-41d5-a315-5146c26a852b-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:17.443298 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:17.443269    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"
Jul 24 14:54:17.443298 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:17.443274    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85" event={"ID":"8ed29299-dfea-41d5-a315-5146c26a852b","Type":"ContainerDied","Data":"d008a8dc44a0338e33714de80796ba39ee2aa7a3c6c7e9b087446aa497961b3a"}
Jul 24 14:54:17.443517 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:17.443338    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d008a8dc44a0338e33714de80796ba39ee2aa7a3c6c7e9b087446aa497961b3a"
Jul 24 14:54:27.250684 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:27.250650    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:54:28.899515 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:28.899474    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"]
Jul 24 14:54:28.903622 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:28.903599    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ab2fb2a0-7tv85"]
Jul 24 14:54:29.049554 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:29.049522    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"]
Jul 24 14:54:29.049803 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:29.049757    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy" containerID="cri-o://abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53" gracePeriod=30
Jul 24 14:54:29.671004 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:29.670974    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ed29299-dfea-41d5-a315-5146c26a852b" path="/var/lib/kubelet/pods/8ed29299-dfea-41d5-a315-5146c26a852b/volumes"
Jul 24 14:54:31.151525 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:31.151484    2584 patch_prober.go:28] interesting pod/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 14:54:31.151964 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:31.151551    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:54:34.308718 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.308661    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:54:34.431234 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431208    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-token\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431396 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431248    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-podinfo\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431396 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431287    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-certs\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431501 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431465    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istiod-ca-cert\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431573 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431535    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-socket\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431573 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431544    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:54:34.431682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431570    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-credential-socket\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431621    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-envoy\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431682 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431662    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-data\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431868 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431688    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnl57\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-kube-api-access-dnl57\") pod \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\" (UID: \"0608695b-fb0c-4341-8fd0-91a6ec6d0b87\") "
Jul 24 14:54:34.431942 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431858    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:54:34.431942 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.431929    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:54:34.432080 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.432060    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.432143 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.432084    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.432143 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.432099    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.432332 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.432298    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:54:34.432850 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.432807    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-data" (OuterVolumeSpecName: "istio-data") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:54:34.433609 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.433587    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 14:54:34.433920 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.433817    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-kube-api-access-dnl57" (OuterVolumeSpecName: "kube-api-access-dnl57") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "kube-api-access-dnl57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:54:34.434031 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.433913    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:54:34.434031 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.433936    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-token" (OuterVolumeSpecName: "istio-token") pod "0608695b-fb0c-4341-8fd0-91a6ec6d0b87" (UID: "0608695b-fb0c-4341-8fd0-91a6ec6d0b87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:54:34.538818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.534133    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.538818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.534166    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.538818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.534181    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.538818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.534194    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnl57\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-kube-api-access-dnl57\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.538818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.534213    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.538818 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.534300    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0608695b-fb0c-4341-8fd0-91a6ec6d0b87-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:54:34.540495 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.540465    2584 generic.go:356] "Generic (PLEG): container finished" podID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerID="abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53" exitCode=0
Jul 24 14:54:34.540629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.540516    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"
Jul 24 14:54:34.540629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.540545    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" event={"ID":"0608695b-fb0c-4341-8fd0-91a6ec6d0b87","Type":"ContainerDied","Data":"abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53"}
Jul 24 14:54:34.540629 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.540620    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k" event={"ID":"0608695b-fb0c-4341-8fd0-91a6ec6d0b87","Type":"ContainerDied","Data":"8c6fa8aa4d2baf5dc5ca1f311d0bd86b0c35ae969c2e9a6b88e55d532c269da8"}
Jul 24 14:54:34.540776 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.540649    2584 scope.go:117] "RemoveContainer" containerID="abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53"
Jul 24 14:54:34.550089 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.550075    2584 scope.go:117] "RemoveContainer" containerID="e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"
Jul 24 14:54:34.557649 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.557636    2584 scope.go:117] "RemoveContainer" containerID="abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53"
Jul 24 14:54:34.557880 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:54:34.557856    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53\": container with ID starting with abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53 not found: ID does not exist" containerID="abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53"
Jul 24 14:54:34.557951 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.557882    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53"} err="failed to get container status \"abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53\": rpc error: code = NotFound desc = could not find container \"abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53\": container with ID starting with abab425a18eef344b511f3b75bdd872ef6f2286a127fc7ad95ab87145252ec53 not found: ID does not exist"
Jul 24 14:54:34.557951 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.557897    2584 scope.go:117] "RemoveContainer" containerID="e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"
Jul 24 14:54:34.558106 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:54:34.558085    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e\": container with ID starting with e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e not found: ID does not exist" containerID="e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"
Jul 24 14:54:34.558168 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.558109    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e"} err="failed to get container status \"e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e\": rpc error: code = NotFound desc = could not find container \"e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e\": container with ID starting with e9e245ec3ec0aa38486982e6eb1a702167c0d050611abe412a1087bd18bbb42e not found: ID does not exist"
Jul 24 14:54:34.565145 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.565097    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"]
Jul 24 14:54:34.569378 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:34.569356    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ab2fb2a0-openshift-default-58567bbb84-4f79k"]
Jul 24 14:54:35.670735 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:54:35.670700    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" path="/var/lib/kubelet/pods/0608695b-fb0c-4341-8fd0-91a6ec6d0b87/volumes"
Jul 24 14:55:57.843781 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:55:57.843748    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:56:07.502927 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:07.502887    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:56:12.297438 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297403    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"]
Jul 24 14:56:12.297857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297800    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy"
Jul 24 14:56:12.297857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297811    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy"
Jul 24 14:56:12.297857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297845    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ed29299-dfea-41d5-a315-5146c26a852b" containerName="revoke-keys"
Jul 24 14:56:12.297857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297852    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ed29299-dfea-41d5-a315-5146c26a852b" containerName="revoke-keys"
Jul 24 14:56:12.297857 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297860    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy"
Jul 24 14:56:12.298051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297866    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy"
Jul 24 14:56:12.298051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297934    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy"
Jul 24 14:56:12.298051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297942    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ed29299-dfea-41d5-a315-5146c26a852b" containerName="revoke-keys"
Jul 24 14:56:12.298051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297950    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0608695b-fb0c-4341-8fd0-91a6ec6d0b87" containerName="istio-proxy"
Jul 24 14:56:12.298051 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.297956    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="78077318-1fbc-4090-bc5b-8dc2c5db2bb4" containerName="istio-proxy"
Jul 24 14:56:12.301163 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.301146    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.304239 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.304219    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d768479c-openshift-default-dockercfg-2dxv9\""
Jul 24 14:56:12.330481 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.330462    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"]
Jul 24 14:56:12.372671 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372643    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-credential-socket\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.372671 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372674    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/843b2498-2813-4a35-9bad-314c036e1f94-istio-podinfo\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.372877 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372693    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-socket\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.372877 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372730    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-certs\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.372877 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372847    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-data\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.373024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372882    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/843b2498-2813-4a35-9bad-314c036e1f94-istiod-ca-cert\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.373024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9f4\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-kube-api-access-rb9f4\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.373024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.372948    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-envoy\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.373024 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.373007    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-istio-token\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474204 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474171    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-data\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474204 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474207    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/843b2498-2813-4a35-9bad-314c036e1f94-istiod-ca-cert\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474224    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9f4\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-kube-api-access-rb9f4\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474247    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-envoy\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474264    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-istio-token\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474405    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-credential-socket\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474434    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/843b2498-2813-4a35-9bad-314c036e1f94-istio-podinfo\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474467 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474465    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-socket\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474814 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474496    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-certs\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474814 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474775    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-credential-socket\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474819    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-certs\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.474922 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474897    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-data\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.475018 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.474951    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-socket\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.475071 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.475053    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/843b2498-2813-4a35-9bad-314c036e1f94-istiod-ca-cert\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.476533 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.476513    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-envoy\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.476847 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.476816    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/843b2498-2813-4a35-9bad-314c036e1f94-istio-podinfo\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.484198 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.484177    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9f4\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-kube-api-access-rb9f4\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.484478 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.484453    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-istio-token\") pod \"e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") " pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.610765 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.610685    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:12.745980 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.745958    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"]
Jul 24 14:56:12.747824 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:56:12.747800    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843b2498_2813_4a35_9bad_314c036e1f94.slice/crio-30d152b5afbb6ccbf267abba218be4be458961b63bf04ca93cc3f09f45135d93 WatchSource:0}: Error finding container 30d152b5afbb6ccbf267abba218be4be458961b63bf04ca93cc3f09f45135d93: Status 404 returned error can't find the container with id 30d152b5afbb6ccbf267abba218be4be458961b63bf04ca93cc3f09f45135d93
Jul 24 14:56:12.750009 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.749974    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:56:12.750091 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.750037    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:56:12.750091 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:12.750063    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:56:13.039492 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:13.039461    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" event={"ID":"843b2498-2813-4a35-9bad-314c036e1f94","Type":"ContainerStarted","Data":"a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"}
Jul 24 14:56:13.039666 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:13.039517    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" event={"ID":"843b2498-2813-4a35-9bad-314c036e1f94","Type":"ContainerStarted","Data":"30d152b5afbb6ccbf267abba218be4be458961b63bf04ca93cc3f09f45135d93"}
Jul 24 14:56:13.061154 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:13.061099    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" podStartSLOduration=1.06108477 podStartE2EDuration="1.06108477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:13.059279709 +0000 UTC m=+2029.843980174" watchObservedRunningTime="2026-07-24 14:56:13.06108477 +0000 UTC m=+2029.845785232"
Jul 24 14:56:13.611733 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:13.611702    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:13.616351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:13.616329    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:14.044623 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:14.044595    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:14.045378 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:14.045360    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:20.055133 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:20.055098    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:56:28.129594 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.129541    2584 generic.go:356] "Generic (PLEG): container finished" podID="843b2498-2813-4a35-9bad-314c036e1f94" containerID="a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d" exitCode=0
Jul 24 14:56:28.129967 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.129615    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" event={"ID":"843b2498-2813-4a35-9bad-314c036e1f94","Type":"ContainerDied","Data":"a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"}
Jul 24 14:56:28.130059 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.130042    2584 scope.go:117] "RemoveContainer" containerID="a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"
Jul 24 14:56:28.130970 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.130944    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:56:28.131025 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.131016    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:56:28.131062 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.131051    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:56:28.300662 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.300632    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"]
Jul 24 14:56:28.304587 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.304569    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:28.307485 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.307462    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:56:28.307897 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.307877    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:56:28.312773 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.312751    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"]
Jul 24 14:56:28.419955 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.419920    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfae35b-f341-4d16-81e4-1da233eba289-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d768479c-v2fmq\" (UID: \"6bfae35b-f341-4d16-81e4-1da233eba289\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:28.520871 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.520827    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfae35b-f341-4d16-81e4-1da233eba289-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d768479c-v2fmq\" (UID: \"6bfae35b-f341-4d16-81e4-1da233eba289\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:28.521667 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.521643    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfae35b-f341-4d16-81e4-1da233eba289-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d768479c-v2fmq\" (UID: \"6bfae35b-f341-4d16-81e4-1da233eba289\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:28.617519 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.617443    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:28.748315 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:28.748290    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"]
Jul 24 14:56:28.750145 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:56:28.750117    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bfae35b_f341_4d16_81e4_1da233eba289.slice/crio-ae21697728fb600960ad8fbb7c1cfbf44bda84bfe943c8a1ccf4bce03d59443f WatchSource:0}: Error finding container ae21697728fb600960ad8fbb7c1cfbf44bda84bfe943c8a1ccf4bce03d59443f: Status 404 returned error can't find the container with id ae21697728fb600960ad8fbb7c1cfbf44bda84bfe943c8a1ccf4bce03d59443f
Jul 24 14:56:29.044799 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.044772    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:29.136909 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.136875    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq" event={"ID":"6bfae35b-f341-4d16-81e4-1da233eba289","Type":"ContainerStarted","Data":"53d04aa646ef4758ce78e5ef43b4d520b82384ee9f96f428267d12d2f3e8c5a0"}
Jul 24 14:56:29.137345 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.136930    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq" event={"ID":"6bfae35b-f341-4d16-81e4-1da233eba289","Type":"ContainerStarted","Data":"ae21697728fb600960ad8fbb7c1cfbf44bda84bfe943c8a1ccf4bce03d59443f"}
Jul 24 14:56:29.138865 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.138816    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" event={"ID":"843b2498-2813-4a35-9bad-314c036e1f94","Type":"ContainerStarted","Data":"ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605"}
Jul 24 14:56:29.189493 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.189382    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq" podStartSLOduration=1.189348774 podStartE2EDuration="1.189348774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:29.160656014 +0000 UTC m=+2045.945356477" watchObservedRunningTime="2026-07-24 14:56:29.189348774 +0000 UTC m=+2045.974049240"
Jul 24 14:56:29.611315 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.611124    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:29.612694 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.612661    2584 patch_prober.go:28] interesting pod/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 24 14:56:29.612812 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:29.612721    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 24 14:56:30.145987 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:30.145952    2584 generic.go:356] "Generic (PLEG): container finished" podID="6bfae35b-f341-4d16-81e4-1da233eba289" containerID="53d04aa646ef4758ce78e5ef43b4d520b82384ee9f96f428267d12d2f3e8c5a0" exitCode=0
Jul 24 14:56:30.146461 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:30.146016    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq" event={"ID":"6bfae35b-f341-4d16-81e4-1da233eba289","Type":"ContainerDied","Data":"53d04aa646ef4758ce78e5ef43b4d520b82384ee9f96f428267d12d2f3e8c5a0"}
Jul 24 14:56:30.611515 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:30.611476    2584 patch_prober.go:28] interesting pod/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 24 14:56:30.611696 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:30.611538    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 24 14:56:31.319327 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:31.319305    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:31.446901 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:31.446798    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfae35b-f341-4d16-81e4-1da233eba289-maas-api-service-ca\") pod \"6bfae35b-f341-4d16-81e4-1da233eba289\" (UID: \"6bfae35b-f341-4d16-81e4-1da233eba289\") "
Jul 24 14:56:31.447120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:31.447098    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bfae35b-f341-4d16-81e4-1da233eba289-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6bfae35b-f341-4d16-81e4-1da233eba289" (UID: "6bfae35b-f341-4d16-81e4-1da233eba289"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:56:31.547585 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:31.547553    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfae35b-f341-4d16-81e4-1da233eba289-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:31.612059 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:31.612021    2584 patch_prober.go:28] interesting pod/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 24 14:56:31.612232 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:31.612077    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 24 14:56:32.158076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:32.158044    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq" event={"ID":"6bfae35b-f341-4d16-81e4-1da233eba289","Type":"ContainerDied","Data":"ae21697728fb600960ad8fbb7c1cfbf44bda84bfe943c8a1ccf4bce03d59443f"}
Jul 24 14:56:32.158076 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:32.158078    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"
Jul 24 14:56:32.158293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:32.158092    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae21697728fb600960ad8fbb7c1cfbf44bda84bfe943c8a1ccf4bce03d59443f"
Jul 24 14:56:32.611950 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:32.611906    2584 patch_prober.go:28] interesting pod/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 24 14:56:32.612357 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:32.611968    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 24 14:56:33.615161 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:33.615133    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:33.615650 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:33.615523    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:33.616174 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:33.616153    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:41.849471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:41.849440    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:56:43.389292 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:43.389259    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"]
Jul 24 14:56:43.392790 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:43.392768    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d768479c-v2fmq"]
Jul 24 14:56:43.677230 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:43.677149    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfae35b-f341-4d16-81e4-1da233eba289" path="/var/lib/kubelet/pods/6bfae35b-f341-4d16-81e4-1da233eba289/volumes"
Jul 24 14:56:51.244736 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:51.244704    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"]
Jul 24 14:56:51.245280 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:51.244933    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy" containerID="cri-o://ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605" gracePeriod=30
Jul 24 14:56:56.504545 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.504486    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:56.573805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.573776    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-socket\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.573805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.573811    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/843b2498-2813-4a35-9bad-314c036e1f94-istiod-ca-cert\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574044 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.573868    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb9f4\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-kube-api-access-rb9f4\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574044 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.573886    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-envoy\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574044 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.573915    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-credential-socket\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574044 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.573956    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-certs\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574044 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574024    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-istio-token\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574051    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/843b2498-2813-4a35-9bad-314c036e1f94-istio-podinfo\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574060    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:56:56.574293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574073    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-data\") pod \"843b2498-2813-4a35-9bad-314c036e1f94\" (UID: \"843b2498-2813-4a35-9bad-314c036e1f94\") "
Jul 24 14:56:56.574293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574276    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:56:56.574490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574283    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843b2498-2813-4a35-9bad-314c036e1f94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:56:56.574490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574414    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.574490 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574472    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.574599 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.574536    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:56:56.575247 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.575222    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-data" (OuterVolumeSpecName: "istio-data") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:56:56.576193 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.576164    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-kube-api-access-rb9f4" (OuterVolumeSpecName: "kube-api-access-rb9f4") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "kube-api-access-rb9f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:56:56.576290 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.576191    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:56:56.576439 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.576425    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-istio-token" (OuterVolumeSpecName: "istio-token") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:56:56.576500 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.576445    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/843b2498-2813-4a35-9bad-314c036e1f94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "843b2498-2813-4a35-9bad-314c036e1f94" (UID: "843b2498-2813-4a35-9bad-314c036e1f94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 14:56:56.675536 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675507    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb9f4\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-kube-api-access-rb9f4\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.675536 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675531    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.675536 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675542    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.675773 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675551    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/843b2498-2813-4a35-9bad-314c036e1f94-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.675773 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675559    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/843b2498-2813-4a35-9bad-314c036e1f94-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.675773 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675566    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/843b2498-2813-4a35-9bad-314c036e1f94-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:56.675773 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:56.675573    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/843b2498-2813-4a35-9bad-314c036e1f94-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:56:57.291755 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.291724    2584 generic.go:356] "Generic (PLEG): container finished" podID="843b2498-2813-4a35-9bad-314c036e1f94" containerID="ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605" exitCode=0
Jul 24 14:56:57.291981 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.291771    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"
Jul 24 14:56:57.291981 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.291810    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" event={"ID":"843b2498-2813-4a35-9bad-314c036e1f94","Type":"ContainerDied","Data":"ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605"}
Jul 24 14:56:57.291981 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.291888    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7" event={"ID":"843b2498-2813-4a35-9bad-314c036e1f94","Type":"ContainerDied","Data":"30d152b5afbb6ccbf267abba218be4be458961b63bf04ca93cc3f09f45135d93"}
Jul 24 14:56:57.291981 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.291926    2584 scope.go:117] "RemoveContainer" containerID="ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605"
Jul 24 14:56:57.308740 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.308722    2584 scope.go:117] "RemoveContainer" containerID="a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"
Jul 24 14:56:57.316568 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.316550    2584 scope.go:117] "RemoveContainer" containerID="ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605"
Jul 24 14:56:57.316810 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:56:57.316792    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605\": container with ID starting with ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605 not found: ID does not exist" containerID="ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605"
Jul 24 14:56:57.316899 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.316818    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605"} err="failed to get container status \"ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605\": rpc error: code = NotFound desc = could not find container \"ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605\": container with ID starting with ba88e6fbfc40e932e8ff77b3d4a93d0f4f9af60504595559f090504dad09f605 not found: ID does not exist"
Jul 24 14:56:57.316899 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.316849    2584 scope.go:117] "RemoveContainer" containerID="a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"
Jul 24 14:56:57.317073 ip-10-0-135-93 kubenswrapper[2584]: E0724 14:56:57.317055    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d\": container with ID starting with a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d not found: ID does not exist" containerID="a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"
Jul 24 14:56:57.317114 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.317081    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d"} err="failed to get container status \"a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d\": rpc error: code = NotFound desc = could not find container \"a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d\": container with ID starting with a68075da1f57bd3b87672672f47bacb11d5bc315392b8d46bb31493e51a69f2d not found: ID does not exist"
Jul 24 14:56:57.321219 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.321197    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"]
Jul 24 14:56:57.326270 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.326252    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d768479c-openshift-default-85fdfc7844-tlsn7"]
Jul 24 14:56:57.670763 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:56:57.670682    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="843b2498-2813-4a35-9bad-314c036e1f94" path="/var/lib/kubelet/pods/843b2498-2813-4a35-9bad-314c036e1f94/volumes"
Jul 24 14:57:13.267445 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267413    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"]
Jul 24 14:57:13.267864 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267790    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bfae35b-f341-4d16-81e4-1da233eba289" containerName="revoke-keys"
Jul 24 14:57:13.267864 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267801    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfae35b-f341-4d16-81e4-1da233eba289" containerName="revoke-keys"
Jul 24 14:57:13.267864 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267809    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy"
Jul 24 14:57:13.267864 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267814    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy"
Jul 24 14:57:13.267864 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267848    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy"
Jul 24 14:57:13.267864 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267857    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy"
Jul 24 14:57:13.268099 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267925    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy"
Jul 24 14:57:13.268099 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.267937    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bfae35b-f341-4d16-81e4-1da233eba289" containerName="revoke-keys"
Jul 24 14:57:13.271107 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.271089    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.274343 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.274323    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0ac5a605-openshift-default-dockercfg-jsshd\""
Jul 24 14:57:13.294350 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.294324    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"]
Jul 24 14:57:13.314188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314162    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-istio-token\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314203    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-certs\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314232    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-socket\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314391 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314299    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f90571-7a63-46a3-9acf-8764688392f4-istio-podinfo\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314391 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314322    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prprn\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-kube-api-access-prprn\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314395    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-envoy\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314471 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314442    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-credential-socket\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314491    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-data\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.314551 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.314511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f90571-7a63-46a3-9acf-8764688392f4-istiod-ca-cert\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415525 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415497    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-data\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415525 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f90571-7a63-46a3-9acf-8764688392f4-istiod-ca-cert\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415558    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-istio-token\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-certs\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415615    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-socket\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415650    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f90571-7a63-46a3-9acf-8764688392f4-istio-podinfo\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415679    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prprn\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-kube-api-access-prprn\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-envoy\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.415775 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.415746    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-credential-socket\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.416188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.416042    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-data\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.416188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.416078    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-certs\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.416188 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.416168    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-socket\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.416351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.416331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-credential-socket\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.416351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.416342    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f90571-7a63-46a3-9acf-8764688392f4-istiod-ca-cert\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.418355 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.418331    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-envoy\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.418532 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.418511    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f90571-7a63-46a3-9acf-8764688392f4-istio-podinfo\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.425479 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.425455    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-istio-token\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.425733 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.425709    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prprn\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-kube-api-access-prprn\") pod \"e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") " pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.582294 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.582208    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:13.733905 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.733882    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"]
Jul 24 14:57:13.736258 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:57:13.736226    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f90571_7a63_46a3_9acf_8764688392f4.slice/crio-aaac5928656b9aed41600aa9aebd71f2a6030749133c3b72fb197a2df74c657a WatchSource:0}: Error finding container aaac5928656b9aed41600aa9aebd71f2a6030749133c3b72fb197a2df74c657a: Status 404 returned error can't find the container with id aaac5928656b9aed41600aa9aebd71f2a6030749133c3b72fb197a2df74c657a
Jul 24 14:57:13.738356 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.738327    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:57:13.738426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.738386    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:57:13.738426 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:13.738416    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:57:14.381882 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:14.381826    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" event={"ID":"d7f90571-7a63-46a3-9acf-8764688392f4","Type":"ContainerStarted","Data":"f559979b64636d106e9baa4e5d7e4dee13a01ed9ac9ebaebcc81ea1f6ea58014"}
Jul 24 14:57:14.382254 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:14.381902    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" event={"ID":"d7f90571-7a63-46a3-9acf-8764688392f4","Type":"ContainerStarted","Data":"aaac5928656b9aed41600aa9aebd71f2a6030749133c3b72fb197a2df74c657a"}
Jul 24 14:57:14.409030 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:14.408980    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podStartSLOduration=1.408966236 podStartE2EDuration="1.408966236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:14.408325599 +0000 UTC m=+2091.193026075" watchObservedRunningTime="2026-07-24 14:57:14.408966236 +0000 UTC m=+2091.193666701"
Jul 24 14:57:14.583322 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:14.583291    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:14.588363 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:14.588339    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:15.386930 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:15.386893    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:15.387825 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:15.387805    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:21.270677 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:21.270645    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:57:23.723756 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:23.723728    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:57:23.727050 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:23.727027    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 14:57:29.472318 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:29.472287    2584 generic.go:356] "Generic (PLEG): container finished" podID="d7f90571-7a63-46a3-9acf-8764688392f4" containerID="f559979b64636d106e9baa4e5d7e4dee13a01ed9ac9ebaebcc81ea1f6ea58014" exitCode=0
Jul 24 14:57:29.472756 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:29.472368    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" event={"ID":"d7f90571-7a63-46a3-9acf-8764688392f4","Type":"ContainerDied","Data":"f559979b64636d106e9baa4e5d7e4dee13a01ed9ac9ebaebcc81ea1f6ea58014"}
Jul 24 14:57:29.472805 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:29.472767    2584 scope.go:117] "RemoveContainer" containerID="f559979b64636d106e9baa4e5d7e4dee13a01ed9ac9ebaebcc81ea1f6ea58014"
Jul 24 14:57:29.473705 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:29.473682    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:57:29.473767 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:29.473743    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:57:29.473802 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:29.473777    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 14:57:30.387913 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:30.387769    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:30.479879 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:30.479823    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" event={"ID":"d7f90571-7a63-46a3-9acf-8764688392f4","Type":"ContainerStarted","Data":"b97e37008eb0570083bf2658062a23f7e030a287d5aa0a7e61346990a6277425"}
Jul 24 14:57:30.583009 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:30.582969    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:30.584351 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:30.584315    2584 patch_prober.go:28] interesting pod/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 24 14:57:30.584470 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:30.584383    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 24 14:57:31.583125 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:31.583092    2584 patch_prober.go:28] interesting pod/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 24 14:57:31.583512 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:31.583151    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 24 14:57:32.583112 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:32.583080    2584 patch_prober.go:28] interesting pod/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 24 14:57:32.583292 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:32.583140    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 24 14:57:33.583452 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:33.583417    2584 patch_prober.go:28] interesting pod/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 24 14:57:33.583890 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:33.583471    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 24 14:57:34.586383 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:34.586353    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:34.586771 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:34.586730    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:34.587474 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:34.587457    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:57:55.454425 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:57:55.454393    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:58:11.851040 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:11.851004    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:58:16.357768 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.357679    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"]
Jul 24 14:58:16.358170 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.358143    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="843b2498-2813-4a35-9bad-314c036e1f94" containerName="istio-proxy"
Jul 24 14:58:16.361203 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.361187    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:16.363503 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.363483    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 14:58:16.363639 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.363528    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 14:58:16.370119 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.370086    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"]
Jul 24 14:58:16.405462 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.405436    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44b95320-91a1-4803-adb3-c900e8c45c8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv\" (UID: \"44b95320-91a1-4803-adb3-c900e8c45c8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:16.506504 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.506479    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44b95320-91a1-4803-adb3-c900e8c45c8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv\" (UID: \"44b95320-91a1-4803-adb3-c900e8c45c8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:16.507089 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.507071    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44b95320-91a1-4803-adb3-c900e8c45c8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv\" (UID: \"44b95320-91a1-4803-adb3-c900e8c45c8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:16.672708 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.672637    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:16.805452 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:16.805417    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"]
Jul 24 14:58:16.807129 ip-10-0-135-93 kubenswrapper[2584]: W0724 14:58:16.807102    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b95320_91a1_4803_adb3_c900e8c45c8e.slice/crio-379125dde35374ecaad44414b0ddb89c67f5a5a9672207cecb780cef561a234f WatchSource:0}: Error finding container 379125dde35374ecaad44414b0ddb89c67f5a5a9672207cecb780cef561a234f: Status 404 returned error can't find the container with id 379125dde35374ecaad44414b0ddb89c67f5a5a9672207cecb780cef561a234f
Jul 24 14:58:17.729120 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:17.729087    2584 generic.go:356] "Generic (PLEG): container finished" podID="44b95320-91a1-4803-adb3-c900e8c45c8e" containerID="46ac1f816302dbf9a3a00fecde4182241a1361b756181fe6dd394318c041d2d6" exitCode=0
Jul 24 14:58:17.729511 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:17.729153    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv" event={"ID":"44b95320-91a1-4803-adb3-c900e8c45c8e","Type":"ContainerDied","Data":"46ac1f816302dbf9a3a00fecde4182241a1361b756181fe6dd394318c041d2d6"}
Jul 24 14:58:17.729511 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:17.729198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv" event={"ID":"44b95320-91a1-4803-adb3-c900e8c45c8e","Type":"ContainerStarted","Data":"379125dde35374ecaad44414b0ddb89c67f5a5a9672207cecb780cef561a234f"}
Jul 24 14:58:18.871275 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:18.871248    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:18.926324 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:18.926298    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44b95320-91a1-4803-adb3-c900e8c45c8e-maas-api-service-ca\") pod \"44b95320-91a1-4803-adb3-c900e8c45c8e\" (UID: \"44b95320-91a1-4803-adb3-c900e8c45c8e\") "
Jul 24 14:58:18.926658 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:18.926631    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44b95320-91a1-4803-adb3-c900e8c45c8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44b95320-91a1-4803-adb3-c900e8c45c8e" (UID: "44b95320-91a1-4803-adb3-c900e8c45c8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:58:19.027459 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:19.027430    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44b95320-91a1-4803-adb3-c900e8c45c8e-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:58:19.739969 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:19.739942    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"
Jul 24 14:58:19.740129 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:19.739990    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv" event={"ID":"44b95320-91a1-4803-adb3-c900e8c45c8e","Type":"ContainerDied","Data":"379125dde35374ecaad44414b0ddb89c67f5a5a9672207cecb780cef561a234f"}
Jul 24 14:58:19.740129 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:19.740039    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="379125dde35374ecaad44414b0ddb89c67f5a5a9672207cecb780cef561a234f"
Jul 24 14:58:23.818007 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:23.817980    2584 scope.go:117] "RemoveContainer" containerID="21a085ce0d9a4978fef5cdf1e9553d9093d42a70d96c0d30e490fdd948bf6f1f"
Jul 24 14:58:30.350447 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:30.350407    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:58:36.458027 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:36.457995    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"]
Jul 24 14:58:36.461771 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:36.461745    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0ac5a605-jvcgv"]
Jul 24 14:58:37.670819 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:37.670786    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44b95320-91a1-4803-adb3-c900e8c45c8e" path="/var/lib/kubelet/pods/44b95320-91a1-4803-adb3-c900e8c45c8e/volumes"
Jul 24 14:58:39.670846 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:58:39.670812    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:59:23.838191 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:23.838152    2584 scope.go:117] "RemoveContainer" containerID="f4c871239531652eafc71d6d75e319ce844738bc8a6c308866196b6f3fb4110c"
Jul 24 14:59:29.909235 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:29.909201    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"]
Jul 24 14:59:29.910321 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:29.909964    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy" containerID="cri-o://b97e37008eb0570083bf2658062a23f7e030a287d5aa0a7e61346990a6277425" gracePeriod=30
Jul 24 14:59:34.156535 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:34.156503    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 14:59:34.589402 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:34.589368    2584 patch_prober.go:28] interesting pod/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 14:59:34.589559 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:34.589431    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 14:59:35.131033 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.131004    2584 generic.go:356] "Generic (PLEG): container finished" podID="d7f90571-7a63-46a3-9acf-8764688392f4" containerID="b97e37008eb0570083bf2658062a23f7e030a287d5aa0a7e61346990a6277425" exitCode=0
Jul 24 14:59:35.131197 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.131078    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" event={"ID":"d7f90571-7a63-46a3-9acf-8764688392f4","Type":"ContainerDied","Data":"b97e37008eb0570083bf2658062a23f7e030a287d5aa0a7e61346990a6277425"}
Jul 24 14:59:35.131197 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.131139    2584 scope.go:117] "RemoveContainer" containerID="f559979b64636d106e9baa4e5d7e4dee13a01ed9ac9ebaebcc81ea1f6ea58014"
Jul 24 14:59:35.171944 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.171892    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:59:35.293572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293545    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-envoy\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293572 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293577    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-data\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293599    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-socket\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293678    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f90571-7a63-46a3-9acf-8764688392f4-istio-podinfo\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293703    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-credential-socket\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293730    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f90571-7a63-46a3-9acf-8764688392f4-istiod-ca-cert\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293755    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-certs\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293780    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prprn\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-kube-api-access-prprn\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.293866 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.293856    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-istio-token\") pod \"d7f90571-7a63-46a3-9acf-8764688392f4\" (UID: \"d7f90571-7a63-46a3-9acf-8764688392f4\") "
Jul 24 14:59:35.294366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294021    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:59:35.294366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294025    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:59:35.294366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294104    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:59:35.294366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294278    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.294366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294299    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.294366 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294315    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.294726 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.294389    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7f90571-7a63-46a3-9acf-8764688392f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 14:59:35.295279 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.295253    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:59:35.295947 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.295921    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7f90571-7a63-46a3-9acf-8764688392f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 14:59:35.296116 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.296093    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-kube-api-access-prprn" (OuterVolumeSpecName: "kube-api-access-prprn") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "kube-api-access-prprn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:59:35.296185 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.296168    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 14:59:35.296301 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.296287    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7f90571-7a63-46a3-9acf-8764688392f4" (UID: "d7f90571-7a63-46a3-9acf-8764688392f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 14:59:35.395445 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.395402    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.395445 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.395440    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7f90571-7a63-46a3-9acf-8764688392f4-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.395445 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.395449    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7f90571-7a63-46a3-9acf-8764688392f4-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.395445 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.395458    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7f90571-7a63-46a3-9acf-8764688392f4-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.395691 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.395466    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prprn\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-kube-api-access-prprn\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:35.395691 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:35.395475    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7f90571-7a63-46a3-9acf-8764688392f4-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 14:59:36.137334 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:36.137306    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"
Jul 24 14:59:36.137502 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:36.137353    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw" event={"ID":"d7f90571-7a63-46a3-9acf-8764688392f4","Type":"ContainerDied","Data":"aaac5928656b9aed41600aa9aebd71f2a6030749133c3b72fb197a2df74c657a"}
Jul 24 14:59:36.137502 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:36.137413    2584 scope.go:117] "RemoveContainer" containerID="b97e37008eb0570083bf2658062a23f7e030a287d5aa0a7e61346990a6277425"
Jul 24 14:59:36.162080 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:36.162056    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"]
Jul 24 14:59:36.168293 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:36.168273    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0ac5a605-openshift-default-58699c6ffb-2sfcw"]
Jul 24 14:59:37.670582 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:37.670550    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" path="/var/lib/kubelet/pods/d7f90571-7a63-46a3-9acf-8764688392f4/volumes"
Jul 24 14:59:43.966384 ip-10-0-135-93 kubenswrapper[2584]: I0724 14:59:43.966352    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:00:00.151975 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.151941    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"]
Jul 24 15:00:00.152471 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.152324    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy"
Jul 24 15:00:00.152471 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.152335    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy"
Jul 24 15:00:00.152471 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.152352    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b95320-91a1-4803-adb3-c900e8c45c8e" containerName="revoke-keys"
Jul 24 15:00:00.152471 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.152357    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b95320-91a1-4803-adb3-c900e8c45c8e" containerName="revoke-keys"
Jul 24 15:00:00.152471 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.152430    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy"
Jul 24 15:00:00.152471 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.152439    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b95320-91a1-4803-adb3-c900e8c45c8e" containerName="revoke-keys"
Jul 24 15:00:00.155354 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.155337    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"
Jul 24 15:00:00.157979 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.157954    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:00:00.163198 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.163174    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"]
Jul 24 15:00:00.165634 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.165618    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"
Jul 24 15:00:00.310051 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:00.305401    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"]
Jul 24 15:00:00.313705 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:00:00.313647    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2d85a6_49ec_458a_afb7_328c899e77d9.slice/crio-9e3f5a50cb49fa836bb0ae84500be72ae0a82cb34ef8231834ba2e1fb74ffb73 WatchSource:0}: Error finding container 9e3f5a50cb49fa836bb0ae84500be72ae0a82cb34ef8231834ba2e1fb74ffb73: Status 404 returned error can't find the container with id 9e3f5a50cb49fa836bb0ae84500be72ae0a82cb34ef8231834ba2e1fb74ffb73
Jul 24 15:00:01.272562 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:01.272537    2584 generic.go:356] "Generic (PLEG): container finished" podID="9f2d85a6-49ec-458a-afb7-328c899e77d9" containerID="1e0683828cd1fcd1659d3a9111ebc0d4ef46b17df1ee7eac338c343ef2fc79a4" exitCode=0
Jul 24 15:00:01.272877 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:01.272611    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8" event={"ID":"9f2d85a6-49ec-458a-afb7-328c899e77d9","Type":"ContainerDied","Data":"1e0683828cd1fcd1659d3a9111ebc0d4ef46b17df1ee7eac338c343ef2fc79a4"}
Jul 24 15:00:01.272877 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:01.272647    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8" event={"ID":"9f2d85a6-49ec-458a-afb7-328c899e77d9","Type":"ContainerStarted","Data":"9e3f5a50cb49fa836bb0ae84500be72ae0a82cb34ef8231834ba2e1fb74ffb73"}
Jul 24 15:00:02.413183 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:02.413156    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"
Jul 24 15:00:03.284760 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:03.284726    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8"
Jul 24 15:00:03.284760 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:03.284744    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748420-zttn8" event={"ID":"9f2d85a6-49ec-458a-afb7-328c899e77d9","Type":"ContainerDied","Data":"9e3f5a50cb49fa836bb0ae84500be72ae0a82cb34ef8231834ba2e1fb74ffb73"}
Jul 24 15:00:03.284996 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:03.284799    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e3f5a50cb49fa836bb0ae84500be72ae0a82cb34ef8231834ba2e1fb74ffb73"
Jul 24 15:00:23.860714 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:23.860678    2584 scope.go:117] "RemoveContainer" containerID="5c80f44d8108400b9914fb7ec140f63cd4a8dd1e93c437c426718b9c99e1bf62"
Jul 24 15:00:23.869173 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:23.869154    2584 scope.go:117] "RemoveContainer" containerID="f8a625b8e33211a86ebb309c547c67020c6dd1d6103699567138826e1ff955a7"
Jul 24 15:00:28.554138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554104    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"]
Jul 24 15:00:28.554665 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554651    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy"
Jul 24 15:00:28.554708 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554667    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy"
Jul 24 15:00:28.554708 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554692    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f2d85a6-49ec-458a-afb7-328c899e77d9" containerName="cleanup"
Jul 24 15:00:28.554708 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554700    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f2d85a6-49ec-458a-afb7-328c899e77d9" containerName="cleanup"
Jul 24 15:00:28.554803 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554782    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f2d85a6-49ec-458a-afb7-328c899e77d9" containerName="cleanup"
Jul 24 15:00:28.554803 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.554795    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f90571-7a63-46a3-9acf-8764688392f4" containerName="istio-proxy"
Jul 24 15:00:28.559541 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.559523    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.562085 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.562068    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-51d655-openshift-default-dockercfg-vrvrp\""
Jul 24 15:00:28.568892 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.568865    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"]
Jul 24 15:00:28.643610 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643583    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-envoy\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643610 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643616    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-credential-socket\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643795 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643637    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-socket\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643795 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-data\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643795 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643705    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-podinfo\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643795 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643722    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-token\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643795 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-certs\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.643795 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643766    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpd4j\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-kube-api-access-kpd4j\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.644042 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.643802    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istiod-ca-cert\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.744729 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.744686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istiod-ca-cert\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.744933 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.744796    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-envoy\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.744933 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.744844    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-credential-socket\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.744933 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.744876    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-socket\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.744933 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.744906    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-data\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745166 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.744943    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-podinfo\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745350 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745323    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-socket\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745435 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745352    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-data\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745526 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745502    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-token\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745627 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745605    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istiod-ca-cert\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745768 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745724    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-credential-socket\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745953 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745908    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-certs\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.745953 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.745951    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpd4j\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-kube-api-access-kpd4j\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.746220 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.746204    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-certs\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.748013 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.747973    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-podinfo\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.748171 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.748149    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-envoy\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.758497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.758468    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-token\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.765977 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.765955    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpd4j\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-kube-api-access-kpd4j\") pod \"e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") " pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:28.871060 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:28.871003    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:29.010159 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.010135    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"]
Jul 24 15:00:29.011470 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:00:29.011443    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bca418d_599b_4c1c_bafe_d5bfe6e89747.slice/crio-20f9167e86f22b1b61f71a21a20e632d19829010594cb304bf3d222322eba39e WatchSource:0}: Error finding container 20f9167e86f22b1b61f71a21a20e632d19829010594cb304bf3d222322eba39e: Status 404 returned error can't find the container with id 20f9167e86f22b1b61f71a21a20e632d19829010594cb304bf3d222322eba39e
Jul 24 15:00:29.013566 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.013535    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:29.013661 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.013642    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:29.013731 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.013690    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:29.419550 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.419518    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" event={"ID":"0bca418d-599b-4c1c-bafe-d5bfe6e89747","Type":"ContainerStarted","Data":"0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"}
Jul 24 15:00:29.419746 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.419568    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" event={"ID":"0bca418d-599b-4c1c-bafe-d5bfe6e89747","Type":"ContainerStarted","Data":"20f9167e86f22b1b61f71a21a20e632d19829010594cb304bf3d222322eba39e"}
Jul 24 15:00:29.474388 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.474342    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podStartSLOduration=1.474325897 podStartE2EDuration="1.474325897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:00:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:00:29.473048347 +0000 UTC m=+2286.257748811" watchObservedRunningTime="2026-07-24 15:00:29.474325897 +0000 UTC m=+2286.259026361"
Jul 24 15:00:29.871820 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:29.871789    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:30.876408 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:30.876379    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:31.429429 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:31.429397    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:31.430365 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:31.430346    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:36.351221 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.351186    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:00:36.723069 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.722985    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"]
Jul 24 15:00:36.727676 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.727655    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.735624 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.735603    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ba2a78-openshift-default-dockercfg-qdw64\""
Jul 24 15:00:36.749981 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.749961    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"]
Jul 24 15:00:36.821327 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821285    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-token\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821483 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821335    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-podinfo\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821483 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371bf6cc-5cce-418d-a6de-e6da705dac98-istiod-ca-cert\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821483 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821431    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-socket\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821483 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821456    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-envoy\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821707 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821569    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-data\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821707 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821616    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4pwr\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-kube-api-access-r4pwr\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821713    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-credential-socket\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.821815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.821740    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-certs\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.922790 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.922750    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-socket\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.922990 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.922795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-envoy\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.922990 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.922862    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-data\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.922990 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.922898    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4pwr\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-kube-api-access-r4pwr\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.922990 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.922964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-credential-socket\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923227 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.922992    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-certs\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923227 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923021    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-token\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923227 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923049    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-podinfo\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923227 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923078    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371bf6cc-5cce-418d-a6de-e6da705dac98-istiod-ca-cert\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923227 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923217    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-socket\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923500 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923300    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-data\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923500 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923363    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-credential-socket\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923591 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923506    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-certs\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.923757 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.923737    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371bf6cc-5cce-418d-a6de-e6da705dac98-istiod-ca-cert\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.925294 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.925275    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-envoy\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.925519 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.925500    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-podinfo\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.934251 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.934227    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-token\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:36.942181 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:36.942159    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4pwr\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-kube-api-access-r4pwr\") pod \"e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") " pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:37.038646 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.038622    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:37.182768 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.182748    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"]
Jul 24 15:00:37.185140 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:00:37.185103    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod371bf6cc_5cce_418d_a6de_e6da705dac98.slice/crio-72228f0afbfdf0a729cdc9e836d86e416d0317178c2e503c23c26f1ad1582879 WatchSource:0}: Error finding container 72228f0afbfdf0a729cdc9e836d86e416d0317178c2e503c23c26f1ad1582879: Status 404 returned error can't find the container with id 72228f0afbfdf0a729cdc9e836d86e416d0317178c2e503c23c26f1ad1582879
Jul 24 15:00:37.187480 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.187444    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:37.187572 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.187501    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:37.187572 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.187532    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:37.468972 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.468883    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" event={"ID":"371bf6cc-5cce-418d-a6de-e6da705dac98","Type":"ContainerStarted","Data":"af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"}
Jul 24 15:00:37.468972 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.468966    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" event={"ID":"371bf6cc-5cce-418d-a6de-e6da705dac98","Type":"ContainerStarted","Data":"72228f0afbfdf0a729cdc9e836d86e416d0317178c2e503c23c26f1ad1582879"}
Jul 24 15:00:37.493330 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:37.493273    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podStartSLOduration=1.493253293 podStartE2EDuration="1.493253293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:00:37.490290129 +0000 UTC m=+2294.274990594" watchObservedRunningTime="2026-07-24 15:00:37.493253293 +0000 UTC m=+2294.277953759"
Jul 24 15:00:38.039311 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:38.039278    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:38.044275 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:38.044249    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:38.473973 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:38.473889    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:38.474895 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:38.474874    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:45.517180 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:45.517141    2584 generic.go:356] "Generic (PLEG): container finished" podID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerID="0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904" exitCode=0
Jul 24 15:00:45.517721 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:45.517214    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" event={"ID":"0bca418d-599b-4c1c-bafe-d5bfe6e89747","Type":"ContainerDied","Data":"0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"}
Jul 24 15:00:45.517721 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:45.517603    2584 scope.go:117] "RemoveContainer" containerID="0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"
Jul 24 15:00:45.518640 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:45.518615    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:45.518738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:45.518683    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:45.518738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:45.518717    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:46.430067 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:46.430035    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:46.525268 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:46.525237    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" event={"ID":"0bca418d-599b-4c1c-bafe-d5bfe6e89747","Type":"ContainerStarted","Data":"22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba"}
Jul 24 15:00:46.871382 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:46.871347    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:46.872619 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:46.872592    2584 patch_prober.go:28] interesting pod/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:00:46.872760 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:46.872643    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 24 15:00:47.871500 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:47.871468    2584 patch_prober.go:28] interesting pod/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:00:47.871905 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:47.871532    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 24 15:00:48.871740 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:48.871703    2584 patch_prober.go:28] interesting pod/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:00:48.872194 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:48.871760    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 24 15:00:49.872105 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:49.872076    2584 patch_prober.go:28] interesting pod/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:00:49.872468 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:49.872131    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 24 15:00:50.872405 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:50.872356    2584 patch_prober.go:28] interesting pod/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 24 15:00:50.872828 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:50.872431    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 24 15:00:51.879702 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:51.879670    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:51.880158 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:51.880107    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:51.880809 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:51.880789    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:00:52.564883 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.564798    2584 generic.go:356] "Generic (PLEG): container finished" podID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerID="af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1" exitCode=0
Jul 24 15:00:52.565093 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.564879    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" event={"ID":"371bf6cc-5cce-418d-a6de-e6da705dac98","Type":"ContainerDied","Data":"af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"}
Jul 24 15:00:52.565404 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.565386    2584 scope.go:117] "RemoveContainer" containerID="af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"
Jul 24 15:00:52.566379 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.566353    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:52.566474 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.566419    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:52.566474 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.566452    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:00:52.851370 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:52.851337    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:00:53.474321 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:53.474294    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:53.572263 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:53.572234    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" event={"ID":"371bf6cc-5cce-418d-a6de-e6da705dac98","Type":"ContainerStarted","Data":"080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5"}
Jul 24 15:00:54.039701 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:54.039666    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:54.040665 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:54.040632    2584 patch_prober.go:28] interesting pod/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 24 15:00:54.040777 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:54.040714    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 24 15:00:55.039369 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:55.039332    2584 patch_prober.go:28] interesting pod/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 24 15:00:55.039783 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:55.039389    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 24 15:00:56.039563 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:56.039527    2584 patch_prober.go:28] interesting pod/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 24 15:00:56.039990 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:56.039591    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 24 15:00:57.039073 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:57.039034    2584 patch_prober.go:28] interesting pod/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 24 15:00:57.039254 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:57.039116    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 24 15:00:58.039727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:58.039690    2584 patch_prober.go:28] interesting pod/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 24 15:00:58.040183 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:58.039756    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 24 15:00:59.043441 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:59.043415    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:59.043915 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:59.043854    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:00:59.044460 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:00:59.044442    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:01:05.459560 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:01:05.459525    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:01:30.079555 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:01:30.079513    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:02:10.551272 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.551239    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"]
Jul 24 15:02:10.554756 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.554739    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:10.557587 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.557566    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:02:10.557982 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.557967    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:02:10.568194 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.568173    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"]
Jul 24 15:02:10.675495 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.675463    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0af9871-7c53-41b7-9998-26083ab4e773-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7\" (UID: \"c0af9871-7c53-41b7-9998-26083ab4e773\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:10.776402 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.776368    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0af9871-7c53-41b7-9998-26083ab4e773-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7\" (UID: \"c0af9871-7c53-41b7-9998-26083ab4e773\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:10.777074 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.777055    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0af9871-7c53-41b7-9998-26083ab4e773-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7\" (UID: \"c0af9871-7c53-41b7-9998-26083ab4e773\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:10.865074 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:10.865007    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:11.023514 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:11.023484    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"]
Jul 24 15:02:11.027308 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:02:11.027282    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0af9871_7c53_41b7_9998_26083ab4e773.slice/crio-318a7a76aa45fd34a91f463aa009e4bf7aa31afea4beca0bf5f50409b7dec799 WatchSource:0}: Error finding container 318a7a76aa45fd34a91f463aa009e4bf7aa31afea4beca0bf5f50409b7dec799: Status 404 returned error can't find the container with id 318a7a76aa45fd34a91f463aa009e4bf7aa31afea4beca0bf5f50409b7dec799
Jul 24 15:02:12.023293 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:12.023257    2584 generic.go:356] "Generic (PLEG): container finished" podID="c0af9871-7c53-41b7-9998-26083ab4e773" containerID="b4bb3bb3cc3e33c08bed8a35ec0594bd25fe0babd940dff1286c81ce971d5cdd" exitCode=0
Jul 24 15:02:12.023684 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:12.023297    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7" event={"ID":"c0af9871-7c53-41b7-9998-26083ab4e773","Type":"ContainerDied","Data":"b4bb3bb3cc3e33c08bed8a35ec0594bd25fe0babd940dff1286c81ce971d5cdd"}
Jul 24 15:02:12.023684 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:12.023335    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7" event={"ID":"c0af9871-7c53-41b7-9998-26083ab4e773","Type":"ContainerStarted","Data":"318a7a76aa45fd34a91f463aa009e4bf7aa31afea4beca0bf5f50409b7dec799"}
Jul 24 15:02:13.160445 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:13.160416    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:13.297288 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:13.297218    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0af9871-7c53-41b7-9998-26083ab4e773-maas-api-service-ca\") pod \"c0af9871-7c53-41b7-9998-26083ab4e773\" (UID: \"c0af9871-7c53-41b7-9998-26083ab4e773\") "
Jul 24 15:02:13.297587 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:13.297569    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0af9871-7c53-41b7-9998-26083ab4e773-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0af9871-7c53-41b7-9998-26083ab4e773" (UID: "c0af9871-7c53-41b7-9998-26083ab4e773"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:13.398041 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:13.398007    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0af9871-7c53-41b7-9998-26083ab4e773-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:14.036668 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:14.036640    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"
Jul 24 15:02:14.036860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:14.036674    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7" event={"ID":"c0af9871-7c53-41b7-9998-26083ab4e773","Type":"ContainerDied","Data":"318a7a76aa45fd34a91f463aa009e4bf7aa31afea4beca0bf5f50409b7dec799"}
Jul 24 15:02:14.036860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:14.036723    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="318a7a76aa45fd34a91f463aa009e4bf7aa31afea4beca0bf5f50409b7dec799"
Jul 24 15:02:14.166322 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:14.166290    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:02:23.757943 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:23.757914    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:02:23.762372 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:23.762352    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:02:25.632320 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:25.631294    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"]
Jul 24 15:02:25.636464 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:25.636432    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-51d655-k8jc7"]
Jul 24 15:02:25.670951 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:25.670922    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0af9871-7c53-41b7-9998-26083ab4e773" path="/var/lib/kubelet/pods/c0af9871-7c53-41b7-9998-26083ab4e773/volumes"
Jul 24 15:02:27.148572 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:27.148544    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:02:34.307847 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.307801    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"]
Jul 24 15:02:34.308398 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.308133    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" containerID="cri-o://22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba" gracePeriod=30
Jul 24 15:02:34.742620 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.742546    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"]
Jul 24 15:02:34.742959 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.742946    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0af9871-7c53-41b7-9998-26083ab4e773" containerName="revoke-keys"
Jul 24 15:02:34.743008 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.742960    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0af9871-7c53-41b7-9998-26083ab4e773" containerName="revoke-keys"
Jul 24 15:02:34.743043 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.743032    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0af9871-7c53-41b7-9998-26083ab4e773" containerName="revoke-keys"
Jul 24 15:02:34.746169 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.746152    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:34.749550 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.749533    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:02:34.749648 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.749600    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:02:34.764813 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.764794    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"]
Jul 24 15:02:34.782440 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.782413    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ab7dbc-6697-4013-ab98-e28f3fca7996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t\" (UID: \"97ab7dbc-6697-4013-ab98-e28f3fca7996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:34.883264 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.883237    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ab7dbc-6697-4013-ab98-e28f3fca7996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t\" (UID: \"97ab7dbc-6697-4013-ab98-e28f3fca7996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:34.883809 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:34.883793    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ab7dbc-6697-4013-ab98-e28f3fca7996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t\" (UID: \"97ab7dbc-6697-4013-ab98-e28f3fca7996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:35.056727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:35.056696    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:35.194870 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:35.194819    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"]
Jul 24 15:02:35.198059 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:02:35.198033    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ab7dbc_6697_4013_ab98_e28f3fca7996.slice/crio-ff504b04d9def48f77a2baadd2d2200975d187e738a80e58cb692bb9aaf6c49a WatchSource:0}: Error finding container ff504b04d9def48f77a2baadd2d2200975d187e738a80e58cb692bb9aaf6c49a: Status 404 returned error can't find the container with id ff504b04d9def48f77a2baadd2d2200975d187e738a80e58cb692bb9aaf6c49a
Jul 24 15:02:36.156470 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:36.156374    2584 generic.go:356] "Generic (PLEG): container finished" podID="97ab7dbc-6697-4013-ab98-e28f3fca7996" containerID="41268d9ac034721495ead31e278c1c07af05e9ca57d1a1fcc455b5c7c70247b3" exitCode=0
Jul 24 15:02:36.156470 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:36.156441    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t" event={"ID":"97ab7dbc-6697-4013-ab98-e28f3fca7996","Type":"ContainerDied","Data":"41268d9ac034721495ead31e278c1c07af05e9ca57d1a1fcc455b5c7c70247b3"}
Jul 24 15:02:36.156858 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:36.156490    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t" event={"ID":"97ab7dbc-6697-4013-ab98-e28f3fca7996","Type":"ContainerStarted","Data":"ff504b04d9def48f77a2baadd2d2200975d187e738a80e58cb692bb9aaf6c49a"}
Jul 24 15:02:36.882138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:36.882099    2584 patch_prober.go:28] interesting pod/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 15:02:36.882301 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:36.882149    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:02:37.303529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:37.303503    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:37.403377 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:37.403350    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ab7dbc-6697-4013-ab98-e28f3fca7996-maas-api-service-ca\") pod \"97ab7dbc-6697-4013-ab98-e28f3fca7996\" (UID: \"97ab7dbc-6697-4013-ab98-e28f3fca7996\") "
Jul 24 15:02:37.403689 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:37.403667    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97ab7dbc-6697-4013-ab98-e28f3fca7996-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97ab7dbc-6697-4013-ab98-e28f3fca7996" (UID: "97ab7dbc-6697-4013-ab98-e28f3fca7996"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:37.505026 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:37.504950    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97ab7dbc-6697-4013-ab98-e28f3fca7996-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:38.168735 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:38.168705    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"
Jul 24 15:02:38.168931 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:38.168758    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t" event={"ID":"97ab7dbc-6697-4013-ab98-e28f3fca7996","Type":"ContainerDied","Data":"ff504b04d9def48f77a2baadd2d2200975d187e738a80e58cb692bb9aaf6c49a"}
Jul 24 15:02:38.168931 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:38.168804    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff504b04d9def48f77a2baadd2d2200975d187e738a80e58cb692bb9aaf6c49a"
Jul 24 15:02:39.565194 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.565170    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:02:39.624528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624464    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istiod-ca-cert\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624519    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-podinfo\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624569    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-credential-socket\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624635    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-envoy\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624661    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-data\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624692    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-token\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624723    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-certs\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624762    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpd4j\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-kube-api-access-kpd4j\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.624815 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624807    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-socket\") pod \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\" (UID: \"0bca418d-599b-4c1c-bafe-d5bfe6e89747\") "
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624922    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624954    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.624991    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.625132    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.625153    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.625168    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.625210 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.625168    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:39.625738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.625704    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-data" (OuterVolumeSpecName: "istio-data") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:39.627029 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.626993    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-token" (OuterVolumeSpecName: "istio-token") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:39.627233 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.627210    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:02:39.627344 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.627327    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-kube-api-access-kpd4j" (OuterVolumeSpecName: "kube-api-access-kpd4j") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "kube-api-access-kpd4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:02:39.627406 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.627327    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0bca418d-599b-4c1c-bafe-d5bfe6e89747" (UID: "0bca418d-599b-4c1c-bafe-d5bfe6e89747"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:02:39.726230 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.726202    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.726230 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.726227    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.726230 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.726235    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.726421 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.726244    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpd4j\" (UniqueName: \"kubernetes.io/projected/0bca418d-599b-4c1c-bafe-d5bfe6e89747-kube-api-access-kpd4j\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.726421 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.726252    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bca418d-599b-4c1c-bafe-d5bfe6e89747-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:39.726421 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:39.726261    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bca418d-599b-4c1c-bafe-d5bfe6e89747-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:02:40.179938 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.179901    2584 generic.go:356] "Generic (PLEG): container finished" podID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerID="22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba" exitCode=0
Jul 24 15:02:40.180085 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.179946    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"
Jul 24 15:02:40.180085 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.179954    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" event={"ID":"0bca418d-599b-4c1c-bafe-d5bfe6e89747","Type":"ContainerDied","Data":"22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba"}
Jul 24 15:02:40.180085 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.179997    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj" event={"ID":"0bca418d-599b-4c1c-bafe-d5bfe6e89747","Type":"ContainerDied","Data":"20f9167e86f22b1b61f71a21a20e632d19829010594cb304bf3d222322eba39e"}
Jul 24 15:02:40.180085 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.180021    2584 scope.go:117] "RemoveContainer" containerID="22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba"
Jul 24 15:02:40.191737 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.191677    2584 scope.go:117] "RemoveContainer" containerID="0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"
Jul 24 15:02:40.199738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.199723    2584 scope.go:117] "RemoveContainer" containerID="22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba"
Jul 24 15:02:40.200006 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:02:40.199988    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba\": container with ID starting with 22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba not found: ID does not exist" containerID="22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba"
Jul 24 15:02:40.200058 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.200014    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba"} err="failed to get container status \"22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba\": rpc error: code = NotFound desc = could not find container \"22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba\": container with ID starting with 22ff0ead63206579e94e34040ce81bff219684edf97bef34d817c0b191ebe5ba not found: ID does not exist"
Jul 24 15:02:40.200058 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.200031    2584 scope.go:117] "RemoveContainer" containerID="0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"
Jul 24 15:02:40.200236 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:02:40.200221    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904\": container with ID starting with 0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904 not found: ID does not exist" containerID="0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"
Jul 24 15:02:40.200277 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.200239    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904"} err="failed to get container status \"0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904\": rpc error: code = NotFound desc = could not find container \"0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904\": container with ID starting with 0e60487b60618a318b92d0a0b78de3cddea18767eb1fe8a137285831f4ff0904 not found: ID does not exist"
Jul 24 15:02:40.208401 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.208378    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"]
Jul 24 15:02:40.217551 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:40.217520    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-51d655-openshift-default-58cd48cd6-m6nmj"]
Jul 24 15:02:41.670182 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:41.670148    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" path="/var/lib/kubelet/pods/0bca418d-599b-4c1c-bafe-d5bfe6e89747/volumes"
Jul 24 15:02:48.352699 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:48.352606    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:02:49.844077 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:49.844047    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"]
Jul 24 15:02:49.847823 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:49.847801    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba2a78-29f6t"]
Jul 24 15:02:51.670803 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:51.670768    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97ab7dbc-6697-4013-ab98-e28f3fca7996" path="/var/lib/kubelet/pods/97ab7dbc-6697-4013-ab98-e28f3fca7996/volumes"
Jul 24 15:02:57.769360 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:57.769324    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"]
Jul 24 15:02:57.769749 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:57.769568    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" containerID="cri-o://080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5" gracePeriod=30
Jul 24 15:02:58.741030 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.740999    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"]
Jul 24 15:02:58.741380 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741367    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy"
Jul 24 15:02:58.741452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741381    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy"
Jul 24 15:02:58.741452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741393    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy"
Jul 24 15:02:58.741452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741399    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy"
Jul 24 15:02:58.741452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741408    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97ab7dbc-6697-4013-ab98-e28f3fca7996" containerName="revoke-keys"
Jul 24 15:02:58.741452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741429    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="97ab7dbc-6697-4013-ab98-e28f3fca7996" containerName="revoke-keys"
Jul 24 15:02:58.741701 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741500    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy"
Jul 24 15:02:58.741701 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.741511    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="97ab7dbc-6697-4013-ab98-e28f3fca7996" containerName="revoke-keys"
Jul 24 15:02:58.746052 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.746031    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.748770 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.748751    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2ea794-openshift-default-dockercfg-8z4gb\""
Jul 24 15:02:58.759593 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.759570    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"]
Jul 24 15:02:58.883949 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.883902    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-podinfo\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.883949 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.883954    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3666c6d-c419-4b20-81aa-f962c96096e1-istiod-ca-cert\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884027    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-data\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884134    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz4j4\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-kube-api-access-kz4j4\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884164    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-certs\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884203    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-socket\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884227    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-token\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884258    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-envoy\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.884497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.884369    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-credential-socket\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.985701 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985650    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-data\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.985923 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985754    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz4j4\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-kube-api-access-kz4j4\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.985923 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985786    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-certs\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.985923 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985855    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-socket\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.985923 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985883    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-token\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.985923 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-envoy\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986124 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985964    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-credential-socket\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986124 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.985989    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-podinfo\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986124 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.986015    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3666c6d-c419-4b20-81aa-f962c96096e1-istiod-ca-cert\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986234 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.986126    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-data\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986383 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.986358    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-certs\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986597 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.986573    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-socket\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986664 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.986604    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-credential-socket\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.986819 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.986799    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3666c6d-c419-4b20-81aa-f962c96096e1-istiod-ca-cert\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.988782 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.988759    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-envoy\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.989444 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.989425    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-podinfo\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.999022 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.998949    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-token\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:58.999144 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:58.999122    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz4j4\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-kube-api-access-kz4j4\") pod \"e2e-api-a-2ea794-openshift-default-69d4959588-44k9r\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") " pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:59.046702 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.046668    2584 patch_prober.go:28] interesting pod/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 15:02:59.046913 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.046722    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:02:59.056721 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.056692    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:02:59.217711 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.217686    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"]
Jul 24 15:02:59.220113 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:02:59.220087    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3666c6d_c419_4b20_81aa_f962c96096e1.slice/crio-81f8a446ffb136d45a95bb696798d999f4f7eca98fe9d91bb243f40df6c55322 WatchSource:0}: Error finding container 81f8a446ffb136d45a95bb696798d999f4f7eca98fe9d91bb243f40df6c55322: Status 404 returned error can't find the container with id 81f8a446ffb136d45a95bb696798d999f4f7eca98fe9d91bb243f40df6c55322
Jul 24 15:02:59.222266 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.222237    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:02:59.222329 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.222304    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:02:59.222370 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.222335    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:02:59.278370 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:02:59.278338    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" event={"ID":"f3666c6d-c419-4b20-81aa-f962c96096e1","Type":"ContainerStarted","Data":"81f8a446ffb136d45a95bb696798d999f4f7eca98fe9d91bb243f40df6c55322"}
Jul 24 15:03:00.285103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:00.285061    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" event={"ID":"f3666c6d-c419-4b20-81aa-f962c96096e1","Type":"ContainerStarted","Data":"836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"}
Jul 24 15:03:00.312371 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:00.312321    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podStartSLOduration=2.312305559 podStartE2EDuration="2.312305559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:00.310394725 +0000 UTC m=+2437.095095189" watchObservedRunningTime="2026-07-24 15:03:00.312305559 +0000 UTC m=+2437.097006023"
Jul 24 15:03:01.057044 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:01.056999    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:01.062139 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:01.062111    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:01.291604 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:01.291573    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:01.292528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:01.292508    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:03.031173 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.031115    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:03:03.132616 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132590    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-podinfo\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132625    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-credential-socket\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132645    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-data\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132672    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-token\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132708    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-envoy\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132725    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-socket\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132745    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4pwr\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-kube-api-access-r4pwr\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.132807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132776    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371bf6cc-5cce-418d-a6de-e6da705dac98-istiod-ca-cert\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.133203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.132865    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-certs\") pod \"371bf6cc-5cce-418d-a6de-e6da705dac98\" (UID: \"371bf6cc-5cce-418d-a6de-e6da705dac98\") "
Jul 24 15:03:03.133203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133021    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:03.133203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133053    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:03.133203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133196    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:03.133402 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133277    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.133402 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133301    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.133402 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133316    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.133402 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133386    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/371bf6cc-5cce-418d-a6de-e6da705dac98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:03.133676 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.133653    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-data" (OuterVolumeSpecName: "istio-data") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:03.135122 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.135093    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-token" (OuterVolumeSpecName: "istio-token") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:03.135122 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.135097    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-kube-api-access-r4pwr" (OuterVolumeSpecName: "kube-api-access-r4pwr") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "kube-api-access-r4pwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:03.135296 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.135278    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:03:03.135296 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.135279    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "371bf6cc-5cce-418d-a6de-e6da705dac98" (UID: "371bf6cc-5cce-418d-a6de-e6da705dac98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:03.234010 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.233977    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.234010 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.234003    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.234010 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.234013    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.234243 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.234021    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/371bf6cc-5cce-418d-a6de-e6da705dac98-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.234243 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.234030    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4pwr\" (UniqueName: \"kubernetes.io/projected/371bf6cc-5cce-418d-a6de-e6da705dac98-kube-api-access-r4pwr\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.234243 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.234040    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/371bf6cc-5cce-418d-a6de-e6da705dac98-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:03.306244 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.306114    2584 generic.go:356] "Generic (PLEG): container finished" podID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerID="080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5" exitCode=0
Jul 24 15:03:03.306244 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.306160    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"
Jul 24 15:03:03.306244 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.306198    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" event={"ID":"371bf6cc-5cce-418d-a6de-e6da705dac98","Type":"ContainerDied","Data":"080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5"}
Jul 24 15:03:03.306675 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.306253    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5" event={"ID":"371bf6cc-5cce-418d-a6de-e6da705dac98","Type":"ContainerDied","Data":"72228f0afbfdf0a729cdc9e836d86e416d0317178c2e503c23c26f1ad1582879"}
Jul 24 15:03:03.306675 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.306285    2584 scope.go:117] "RemoveContainer" containerID="080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5"
Jul 24 15:03:03.317294 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.317278    2584 scope.go:117] "RemoveContainer" containerID="af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"
Jul 24 15:03:03.325370 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.325351    2584 scope.go:117] "RemoveContainer" containerID="080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5"
Jul 24 15:03:03.325608 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:03:03.325590    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5\": container with ID starting with 080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5 not found: ID does not exist" containerID="080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5"
Jul 24 15:03:03.325662 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.325618    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5"} err="failed to get container status \"080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5\": rpc error: code = NotFound desc = could not find container \"080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5\": container with ID starting with 080fd292919bab85da748af6977f997a5512dededb86cb282651ed01cf889bb5 not found: ID does not exist"
Jul 24 15:03:03.325662 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.325636    2584 scope.go:117] "RemoveContainer" containerID="af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"
Jul 24 15:03:03.325850 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:03:03.325822    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1\": container with ID starting with af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1 not found: ID does not exist" containerID="af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"
Jul 24 15:03:03.325902 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.325854    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1"} err="failed to get container status \"af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1\": rpc error: code = NotFound desc = could not find container \"af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1\": container with ID starting with af7dbe083e3d1f21c602d2f133dc2e1431b25056474b97cad2f3048d0da954d1 not found: ID does not exist"
Jul 24 15:03:03.340533 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.340511    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"]
Jul 24 15:03:03.346986 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.346968    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ba2a78-openshift-default-59bf5d494b-d8bk5"]
Jul 24 15:03:03.672111 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:03.672025    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" path="/var/lib/kubelet/pods/371bf6cc-5cce-418d-a6de-e6da705dac98/volumes"
Jul 24 15:03:07.776453 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.776420    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"]
Jul 24 15:03:07.776937 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.776901    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy"
Jul 24 15:03:07.776937 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.776915    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy"
Jul 24 15:03:07.776937 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.776939    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy"
Jul 24 15:03:07.777060 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.776944    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy"
Jul 24 15:03:07.777060 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.776997    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy"
Jul 24 15:03:07.777060 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.777007    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bca418d-599b-4c1c-bafe-d5bfe6e89747" containerName="istio-proxy"
Jul 24 15:03:07.781513 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.781494    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.784333 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.784309    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a1a404-openshift-default-dockercfg-t9m84\""
Jul 24 15:03:07.795233 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.795211    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"]
Jul 24 15:03:07.884358 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884312    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istiod-ca-cert\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884560 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884380    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-envoy\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884560 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-data\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884560 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884464    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q578z\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-kube-api-access-q578z\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884560 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884498    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-podinfo\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884791 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-socket\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884791 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-certs\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884791 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884771    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-credential-socket\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.884984 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.884804    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-token\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985485 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985442    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-socket\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985685 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985515    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-certs\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985685 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985543    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-credential-socket\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985685 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-token\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985685 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985603    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istiod-ca-cert\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985685 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985641    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-envoy\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985987 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-data\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985987 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985713    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q578z\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-kube-api-access-q578z\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985987 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985739    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-podinfo\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.985987 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.985952    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-socket\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.986202 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.986015    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-certs\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.986653 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.986626    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istiod-ca-cert\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.986784 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.986708    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-credential-socket\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.986880 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.986779    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-data\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.988511 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.988482    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-podinfo\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.988816 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.988799    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-envoy\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.996013 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.995991    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q578z\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-kube-api-access-q578z\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:07.998960 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:07.998940    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-token\") pod \"e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") " pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:08.062688 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.062617    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:03:08.094239 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.094209    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:08.245353 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.245327    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"]
Jul 24 15:03:08.246749 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:03:08.246727    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d02758d_d4c2_43e8_928a_81c69ab96cc8.slice/crio-e5c85ab96c936a7dae3f19d59d70bcd862f6176041b85a895626639f96c61e21 WatchSource:0}: Error finding container e5c85ab96c936a7dae3f19d59d70bcd862f6176041b85a895626639f96c61e21: Status 404 returned error can't find the container with id e5c85ab96c936a7dae3f19d59d70bcd862f6176041b85a895626639f96c61e21
Jul 24 15:03:08.248917 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.248887    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:08.248969 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.248947    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:08.249008 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.248976    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:08.338416 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.338330    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" event={"ID":"9d02758d-d4c2-43e8-928a-81c69ab96cc8","Type":"ContainerStarted","Data":"729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"}
Jul 24 15:03:08.338416 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.338401    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" event={"ID":"9d02758d-d4c2-43e8-928a-81c69ab96cc8","Type":"ContainerStarted","Data":"e5c85ab96c936a7dae3f19d59d70bcd862f6176041b85a895626639f96c61e21"}
Jul 24 15:03:08.365636 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:08.365583    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podStartSLOduration=1.365566215 podStartE2EDuration="1.365566215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:08.365304143 +0000 UTC m=+2445.150004608" watchObservedRunningTime="2026-07-24 15:03:08.365566215 +0000 UTC m=+2445.150266679"
Jul 24 15:03:09.094983 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:09.094950    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:09.100496 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:09.100469    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:09.343480 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:09.343451    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:09.344413 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:09.344390    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:16.293246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.293204    2584 patch_prober.go:28] interesting pod/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 24 15:03:16.293807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.293289    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 24 15:03:16.382781 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.382742    2584 generic.go:356] "Generic (PLEG): container finished" podID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerID="836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317" exitCode=0
Jul 24 15:03:16.382964 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.382817    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" event={"ID":"f3666c6d-c419-4b20-81aa-f962c96096e1","Type":"ContainerDied","Data":"836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"}
Jul 24 15:03:16.383277 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.383251    2584 scope.go:117] "RemoveContainer" containerID="836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"
Jul 24 15:03:16.385078 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.385042    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:16.385209 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.385136    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:16.385209 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:16.385194    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:17.393590 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:17.393469    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" event={"ID":"f3666c6d-c419-4b20-81aa-f962c96096e1","Type":"ContainerStarted","Data":"f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259"}
Jul 24 15:03:18.057802 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:18.057764    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:18.058385 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:18.058358    2584 patch_prober.go:28] interesting pod/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 24 15:03:18.058604 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:18.058579    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 24 15:03:19.057498 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:19.057457    2584 patch_prober.go:28] interesting pod/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 24 15:03:19.058016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:19.057516    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 24 15:03:20.057644 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:20.057609    2584 patch_prober.go:28] interesting pod/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 24 15:03:20.058127 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:20.057675    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 24 15:03:21.058020 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:21.057978    2584 patch_prober.go:28] interesting pod/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 24 15:03:21.058451 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:21.058036    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 24 15:03:22.061016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:22.060981    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:22.061476 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:22.061370    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:22.061963 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:22.061944    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:23.439352 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.439316    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"]
Jul 24 15:03:23.439785 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.439771    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="371bf6cc-5cce-418d-a6de-e6da705dac98" containerName="istio-proxy"
Jul 24 15:03:23.442814 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.442799    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:23.449072 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.449055    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:03:23.449998 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.449975    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:03:23.454737 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.454713    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"]
Jul 24 15:03:23.474260 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.474239    2584 generic.go:356] "Generic (PLEG): container finished" podID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerID="729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050" exitCode=0
Jul 24 15:03:23.474363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.474318    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" event={"ID":"9d02758d-d4c2-43e8-928a-81c69ab96cc8","Type":"ContainerDied","Data":"729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"}
Jul 24 15:03:23.474983 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.474962    2584 scope.go:117] "RemoveContainer" containerID="729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"
Jul 24 15:03:23.476205 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.476175    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:23.476269 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.476256    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:23.476319 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.476303    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:03:23.534179 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.534149    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156c9225-95b4-4b49-be49-87c779348b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2ea794-z6knc\" (UID: \"156c9225-95b4-4b49-be49-87c779348b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:23.635048 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.635016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156c9225-95b4-4b49-be49-87c779348b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2ea794-z6knc\" (UID: \"156c9225-95b4-4b49-be49-87c779348b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:23.637617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.637595    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:03:23.646263 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.646241    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156c9225-95b4-4b49-be49-87c779348b27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2ea794-z6knc\" (UID: \"156c9225-95b4-4b49-be49-87c779348b27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:23.755516 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.755450    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:03:23.763348 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.763326    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:23.902046 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.902017    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"]
Jul 24 15:03:23.903663 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:03:23.903630    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156c9225_95b4_4b49_be49_87c779348b27.slice/crio-4c2136e8477bbbf21688d2919253d93c1d9a900424f40d804bda8c61abcb9b7a WatchSource:0}: Error finding container 4c2136e8477bbbf21688d2919253d93c1d9a900424f40d804bda8c61abcb9b7a: Status 404 returned error can't find the container with id 4c2136e8477bbbf21688d2919253d93c1d9a900424f40d804bda8c61abcb9b7a
Jul 24 15:03:23.913370 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:23.913347    2584 scope.go:117] "RemoveContainer" containerID="53d04aa646ef4758ce78e5ef43b4d520b82384ee9f96f428267d12d2f3e8c5a0"
Jul 24 15:03:24.343925 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:24.343888    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:24.481418 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:24.481337    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" event={"ID":"9d02758d-d4c2-43e8-928a-81c69ab96cc8","Type":"ContainerStarted","Data":"0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625"}
Jul 24 15:03:24.483092 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:24.483064    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc" event={"ID":"156c9225-95b4-4b49-be49-87c779348b27","Type":"ContainerStarted","Data":"7b547593f15a7096bd859179da2115a573d8946c7665a05c9bfab06da7153b4c"}
Jul 24 15:03:24.483207 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:24.483132    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc" event={"ID":"156c9225-95b4-4b49-be49-87c779348b27","Type":"ContainerStarted","Data":"4c2136e8477bbbf21688d2919253d93c1d9a900424f40d804bda8c61abcb9b7a"}
Jul 24 15:03:24.520103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:24.520062    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc" podStartSLOduration=1.52004761 podStartE2EDuration="1.52004761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:24.518519024 +0000 UTC m=+2461.303219480" watchObservedRunningTime="2026-07-24 15:03:24.52004761 +0000 UTC m=+2461.304748073"
Jul 24 15:03:25.052047 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:25.052011    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:03:25.094744 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:25.094707    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:25.095964 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:25.095930    2584 patch_prober.go:28] interesting pod/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:03:25.096110 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:25.095995    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:03:25.493780 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:25.493677    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc" event={"ID":"156c9225-95b4-4b49-be49-87c779348b27","Type":"ContainerDied","Data":"7b547593f15a7096bd859179da2115a573d8946c7665a05c9bfab06da7153b4c"}
Jul 24 15:03:25.494289 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:25.494082    2584 generic.go:356] "Generic (PLEG): container finished" podID="156c9225-95b4-4b49-be49-87c779348b27" containerID="7b547593f15a7096bd859179da2115a573d8946c7665a05c9bfab06da7153b4c" exitCode=0
Jul 24 15:03:26.095577 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:26.095539    2584 patch_prober.go:28] interesting pod/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:03:26.095917 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:26.095878    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:03:26.672039 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:26.672015    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:26.765634 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:26.765552    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156c9225-95b4-4b49-be49-87c779348b27-maas-api-service-ca\") pod \"156c9225-95b4-4b49-be49-87c779348b27\" (UID: \"156c9225-95b4-4b49-be49-87c779348b27\") "
Jul 24 15:03:26.765892 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:26.765857    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/156c9225-95b4-4b49-be49-87c779348b27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "156c9225-95b4-4b49-be49-87c779348b27" (UID: "156c9225-95b4-4b49-be49-87c779348b27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:26.766045 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:26.766005    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/156c9225-95b4-4b49-be49-87c779348b27-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:27.095292 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:27.095252    2584 patch_prober.go:28] interesting pod/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:03:27.095510 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:27.095320    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:03:27.512241 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:27.512146    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"
Jul 24 15:03:27.512241 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:27.512176    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc" event={"ID":"156c9225-95b4-4b49-be49-87c779348b27","Type":"ContainerDied","Data":"4c2136e8477bbbf21688d2919253d93c1d9a900424f40d804bda8c61abcb9b7a"}
Jul 24 15:03:27.512241 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:27.512234    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c2136e8477bbbf21688d2919253d93c1d9a900424f40d804bda8c61abcb9b7a"
Jul 24 15:03:28.094917 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:28.094877    2584 patch_prober.go:28] interesting pod/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:03:28.095502 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:28.095474    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:03:29.095447 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:29.095413    2584 patch_prober.go:28] interesting pod/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 24 15:03:29.095851 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:29.095475    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 24 15:03:30.098859 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:30.098811    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:30.099280 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:30.099083    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:30.099740 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:30.099720    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:03:41.854544 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:41.854512    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:03:43.567695 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:43.567661    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"]
Jul 24 15:03:43.572925 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:43.572901    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2ea794-z6knc"]
Jul 24 15:03:43.670824 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:43.670791    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="156c9225-95b4-4b49-be49-87c779348b27" path="/var/lib/kubelet/pods/156c9225-95b4-4b49-be49-87c779348b27/volumes"
Jul 24 15:03:51.577917 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:51.577881    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"]
Jul 24 15:03:51.578396 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:51.578099    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" containerID="cri-o://f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259" gracePeriod=30
Jul 24 15:03:52.032180 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.032152    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"]
Jul 24 15:03:52.032605 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.032591    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="156c9225-95b4-4b49-be49-87c779348b27" containerName="revoke-keys"
Jul 24 15:03:52.032689 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.032607    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="156c9225-95b4-4b49-be49-87c779348b27" containerName="revoke-keys"
Jul 24 15:03:52.032689 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.032677    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="156c9225-95b4-4b49-be49-87c779348b27" containerName="revoke-keys"
Jul 24 15:03:52.037039 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.037021    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:52.039916 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.039896    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:03:52.040583 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.040566    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:03:52.046622 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.046604    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"]
Jul 24 15:03:52.063108 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.063086    2584 patch_prober.go:28] interesting pod/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 15:03:52.063199 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.063129    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:03:52.198118 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.198083    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a1a404-4cppn\" (UID: \"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:52.299037 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.298953    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a1a404-4cppn\" (UID: \"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:52.299589 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.299568    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a1a404-4cppn\" (UID: \"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:52.348453 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.348426    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:52.500211 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.500184    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"]
Jul 24 15:03:52.501774 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:03:52.501744    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716e2efe_7bd5_4199_b1fe_5fcfb1c9c4b6.slice/crio-06e94686bdfd390b750f95cc9835cc2b0e0178072d393bdf89905c86c99a5ae4 WatchSource:0}: Error finding container 06e94686bdfd390b750f95cc9835cc2b0e0178072d393bdf89905c86c99a5ae4: Status 404 returned error can't find the container with id 06e94686bdfd390b750f95cc9835cc2b0e0178072d393bdf89905c86c99a5ae4
Jul 24 15:03:52.651356 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:52.651282    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn" event={"ID":"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6","Type":"ContainerStarted","Data":"06e94686bdfd390b750f95cc9835cc2b0e0178072d393bdf89905c86c99a5ae4"}
Jul 24 15:03:53.658531 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:53.658498    2584 generic.go:356] "Generic (PLEG): container finished" podID="716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6" containerID="4de28ddd28bca394458b5695d8d4685ad93c8cd126688c69f0cf8d610abce8b0" exitCode=0
Jul 24 15:03:53.658934 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:53.658536    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn" event={"ID":"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6","Type":"ContainerDied","Data":"4de28ddd28bca394458b5695d8d4685ad93c8cd126688c69f0cf8d610abce8b0"}
Jul 24 15:03:54.803596 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:54.803573    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:54.921790 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:54.921717    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6-maas-api-service-ca\") pod \"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6\" (UID: \"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6\") "
Jul 24 15:03:54.922074 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:54.922052    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6" (UID: "716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:55.022691 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:55.022649    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:55.670913 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:55.670880    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"
Jul 24 15:03:55.671134 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:55.671111    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn" event={"ID":"716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6","Type":"ContainerDied","Data":"06e94686bdfd390b750f95cc9835cc2b0e0178072d393bdf89905c86c99a5ae4"}
Jul 24 15:03:55.671198 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:55.671159    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06e94686bdfd390b750f95cc9835cc2b0e0178072d393bdf89905c86c99a5ae4"
Jul 24 15:03:56.832622 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.832600    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:56.942401 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942328    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-token\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942401 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942364    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-podinfo\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942401 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942380    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz4j4\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-kube-api-access-kz4j4\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942639 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942418    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-envoy\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942639 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942448    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-credential-socket\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942639 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942473    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3666c6d-c419-4b20-81aa-f962c96096e1-istiod-ca-cert\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942639 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942503    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-data\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942639 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942530    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-certs\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942639 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942560    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-socket\") pod \"f3666c6d-c419-4b20-81aa-f962c96096e1\" (UID: \"f3666c6d-c419-4b20-81aa-f962c96096e1\") "
Jul 24 15:03:56.942958 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.942938    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:56.943186 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.943104    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:56.943186 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.943133    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:56.943186 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.943143    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3666c6d-c419-4b20-81aa-f962c96096e1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:03:56.943937 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.943902    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:56.944729 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.944699    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:03:56.944828 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.944793    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:03:56.944828 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.944805    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-kube-api-access-kz4j4" (OuterVolumeSpecName: "kube-api-access-kz4j4") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "kube-api-access-kz4j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:56.944949 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:56.944934    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3666c6d-c419-4b20-81aa-f962c96096e1" (UID: "f3666c6d-c419-4b20-81aa-f962c96096e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:03:57.043298 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043268    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043298 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043292    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043298 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043302    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3666c6d-c419-4b20-81aa-f962c96096e1-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043527 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043310    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043527 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043317    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043527 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043325    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3666c6d-c419-4b20-81aa-f962c96096e1-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043527 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043332    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043527 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043340    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3666c6d-c419-4b20-81aa-f962c96096e1-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.043527 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.043348    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz4j4\" (UniqueName: \"kubernetes.io/projected/f3666c6d-c419-4b20-81aa-f962c96096e1-kube-api-access-kz4j4\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:03:57.683310 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.683285    2584 generic.go:356] "Generic (PLEG): container finished" podID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerID="f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259" exitCode=0
Jul 24 15:03:57.683448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.683341    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"
Jul 24 15:03:57.683448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.683367    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" event={"ID":"f3666c6d-c419-4b20-81aa-f962c96096e1","Type":"ContainerDied","Data":"f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259"}
Jul 24 15:03:57.683448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.683419    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r" event={"ID":"f3666c6d-c419-4b20-81aa-f962c96096e1","Type":"ContainerDied","Data":"81f8a446ffb136d45a95bb696798d999f4f7eca98fe9d91bb243f40df6c55322"}
Jul 24 15:03:57.683448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.683442    2584 scope.go:117] "RemoveContainer" containerID="f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259"
Jul 24 15:03:57.692712 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.692563    2584 scope.go:117] "RemoveContainer" containerID="836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"
Jul 24 15:03:57.700271 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.700256    2584 scope.go:117] "RemoveContainer" containerID="f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259"
Jul 24 15:03:57.700516 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:03:57.700495    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259\": container with ID starting with f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259 not found: ID does not exist" containerID="f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259"
Jul 24 15:03:57.700591 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.700527    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259"} err="failed to get container status \"f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259\": rpc error: code = NotFound desc = could not find container \"f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259\": container with ID starting with f947e1a5eac09eaf32863f5298eff0fea195fa84d84111ef9f23ccdb12b1c259 not found: ID does not exist"
Jul 24 15:03:57.700591 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.700550    2584 scope.go:117] "RemoveContainer" containerID="836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"
Jul 24 15:03:57.700824 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:03:57.700794    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317\": container with ID starting with 836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317 not found: ID does not exist" containerID="836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"
Jul 24 15:03:57.700894 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.700848    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317"} err="failed to get container status \"836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317\": rpc error: code = NotFound desc = could not find container \"836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317\": container with ID starting with 836ea90c26d26e4a52fb0ac3bf8cb553b6f12a70b26469eb90916ba99d846317 not found: ID does not exist"
Jul 24 15:03:57.707727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.707699    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"]
Jul 24 15:03:57.714268 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:57.714251    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2ea794-openshift-default-69d4959588-44k9r"]
Jul 24 15:03:59.670869 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:03:59.670815    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" path="/var/lib/kubelet/pods/f3666c6d-c419-4b20-81aa-f962c96096e1/volumes"
Jul 24 15:04:12.115561 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:12.115525    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"]
Jul 24 15:04:12.118719 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:12.118695    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a1a404-4cppn"]
Jul 24 15:04:12.957533 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:12.957499    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:04:13.671451 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:13.671421    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6" path="/var/lib/kubelet/pods/716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6/volumes"
Jul 24 15:04:19.241761 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:19.241683    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"]
Jul 24 15:04:19.242157 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:19.241951    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy" containerID="cri-o://0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625" gracePeriod=30
Jul 24 15:04:20.276332 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276303    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"]
Jul 24 15:04:20.276779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276675    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy"
Jul 24 15:04:20.276779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276685    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy"
Jul 24 15:04:20.276779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276702    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6" containerName="revoke-keys"
Jul 24 15:04:20.276779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276708    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6" containerName="revoke-keys"
Jul 24 15:04:20.276779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276758    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy"
Jul 24 15:04:20.276779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.276770    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="716e2efe-7bd5-4199-b1fe-5fcfb1c9c4b6" containerName="revoke-keys"
Jul 24 15:04:20.280050 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.280034    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.282338 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.282320    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-876f50-openshift-default-dockercfg-8mfvm\""
Jul 24 15:04:20.295458 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.295433    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"]
Jul 24 15:04:20.342738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.342708    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-envoy\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.342885 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.342758    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-credential-socket\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.342885 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.342871    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxrbf\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-kube-api-access-zxrbf\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.343016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.342935    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-certs\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.343016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.342961    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1376081d-75c9-499c-b274-417f6fab91a0-istiod-ca-cert\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.343016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.342995    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-data\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.343164 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.343020    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1376081d-75c9-499c-b274-417f6fab91a0-istio-podinfo\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.343164 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.343064    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-socket\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.343164 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.343108    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-istio-token\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444513 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-envoy\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444706 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-credential-socket\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444706 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444561    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxrbf\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-kube-api-access-zxrbf\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444706 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444598    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-certs\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444706 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444620    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1376081d-75c9-499c-b274-417f6fab91a0-istiod-ca-cert\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444945 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444740    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-data\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444945 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444772    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1376081d-75c9-499c-b274-417f6fab91a0-istio-podinfo\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444945 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444818    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-socket\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.444945 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.444894    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-istio-token\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.445152 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.445040    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-certs\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.445152 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.445106    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-credential-socket\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.445653 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.445457    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-socket\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.445653 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.445582    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-data\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.445859 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.445788    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1376081d-75c9-499c-b274-417f6fab91a0-istiod-ca-cert\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.447251 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.447224    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-envoy\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.447383 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.447365    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1376081d-75c9-499c-b274-417f6fab91a0-istio-podinfo\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.453713 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.453692    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-istio-token\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.454043 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.454023    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxrbf\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-kube-api-access-zxrbf\") pod \"e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") " pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.590895 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.590800    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:20.753619 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.753530    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"]
Jul 24 15:04:20.756027 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:04:20.755998    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1376081d_75c9_499c_b274_417f6fab91a0.slice/crio-a807a28e403077f1502c8eff05564674ce2b57ad48edd2fc78b41196f13f1321 WatchSource:0}: Error finding container a807a28e403077f1502c8eff05564674ce2b57ad48edd2fc78b41196f13f1321: Status 404 returned error can't find the container with id a807a28e403077f1502c8eff05564674ce2b57ad48edd2fc78b41196f13f1321
Jul 24 15:04:20.758223 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.758194    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:20.758308 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.758261    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:20.758308 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.758292    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:20.808169 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:20.808146    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" event={"ID":"1376081d-75c9-499c-b274-417f6fab91a0","Type":"ContainerStarted","Data":"a807a28e403077f1502c8eff05564674ce2b57ad48edd2fc78b41196f13f1321"}
Jul 24 15:04:21.815614 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:21.815576    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" event={"ID":"1376081d-75c9-499c-b274-417f6fab91a0","Type":"ContainerStarted","Data":"f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"}
Jul 24 15:04:21.840553 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:21.840509    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podStartSLOduration=1.840494962 podStartE2EDuration="1.840494962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:21.837934101 +0000 UTC m=+2518.622634564" watchObservedRunningTime="2026-07-24 15:04:21.840494962 +0000 UTC m=+2518.625195424"
Jul 24 15:04:22.591379 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:22.591347    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:22.595967 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:22.595939    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:22.821874 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:22.821826    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:22.822853 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:22.822818    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:23.948420 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:23.948390    2584 scope.go:117] "RemoveContainer" containerID="46ac1f816302dbf9a3a00fecde4182241a1361b756181fe6dd394318c041d2d6"
Jul 24 15:04:24.506252 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.506195    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:04:24.581015 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.580987    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q578z\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-kube-api-access-q578z\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581163 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581036    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istiod-ca-cert\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581163 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581065    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-token\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581163 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581098    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-credential-socket\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581278 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581211    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-podinfo\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581278 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581234    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-data\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581278 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581262    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-envoy\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581425 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581287    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-socket\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581425 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581316    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-certs\") pod \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\" (UID: \"9d02758d-d4c2-43e8-928a-81c69ab96cc8\") "
Jul 24 15:04:24.581532 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581439    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:04:24.581594 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581483    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:04:24.581701 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581660    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:04:24.581764 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581704    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:04:24.581764 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581711    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.581764 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.581735    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.582189 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.582164    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:04:24.583259 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.583237    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-kube-api-access-q578z" (OuterVolumeSpecName: "kube-api-access-q578z") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "kube-api-access-q578z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:04:24.583345 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.583306    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:04:24.583754 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.583735    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:04:24.583754 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.583739    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d02758d-d4c2-43e8-928a-81c69ab96cc8" (UID: "9d02758d-d4c2-43e8-928a-81c69ab96cc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:04:24.682234 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682207    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.682234 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682231    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.682234 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682241    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.682445 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682249    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.682445 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682256    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.682445 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682264    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d02758d-d4c2-43e8-928a-81c69ab96cc8-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.682445 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.682271    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q578z\" (UniqueName: \"kubernetes.io/projected/9d02758d-d4c2-43e8-928a-81c69ab96cc8-kube-api-access-q578z\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:04:24.833526 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.833496    2584 generic.go:356] "Generic (PLEG): container finished" podID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerID="0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625" exitCode=0
Jul 24 15:04:24.833680 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.833539    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" event={"ID":"9d02758d-d4c2-43e8-928a-81c69ab96cc8","Type":"ContainerDied","Data":"0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625"}
Jul 24 15:04:24.833680 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.833594    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq" event={"ID":"9d02758d-d4c2-43e8-928a-81c69ab96cc8","Type":"ContainerDied","Data":"e5c85ab96c936a7dae3f19d59d70bcd862f6176041b85a895626639f96c61e21"}
Jul 24 15:04:24.833680 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.833624    2584 scope.go:117] "RemoveContainer" containerID="0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625"
Jul 24 15:04:24.833820 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.833553    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"
Jul 24 15:04:24.844689 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.844670    2584 scope.go:117] "RemoveContainer" containerID="729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"
Jul 24 15:04:24.853424 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.853402    2584 scope.go:117] "RemoveContainer" containerID="0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625"
Jul 24 15:04:24.853689 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:04:24.853671    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625\": container with ID starting with 0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625 not found: ID does not exist" containerID="0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625"
Jul 24 15:04:24.853738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.853698    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625"} err="failed to get container status \"0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625\": rpc error: code = NotFound desc = could not find container \"0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625\": container with ID starting with 0c6ffaa53f6cf8cdca7e11a7aa7d8ead1819cda6d0f1f82f7c962e023083f625 not found: ID does not exist"
Jul 24 15:04:24.853738 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.853715    2584 scope.go:117] "RemoveContainer" containerID="729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"
Jul 24 15:04:24.854059 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:04:24.854041    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050\": container with ID starting with 729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050 not found: ID does not exist" containerID="729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"
Jul 24 15:04:24.854138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.854063    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050"} err="failed to get container status \"729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050\": rpc error: code = NotFound desc = could not find container \"729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050\": container with ID starting with 729ad48998822a64014dd3c2c4ab2e431de6b3ee9cc0355475415ef600e27050 not found: ID does not exist"
Jul 24 15:04:24.861052 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.861033    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"]
Jul 24 15:04:24.869106 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:24.869088    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a1a404-openshift-default-5c5c54cdd8-j95tq"]
Jul 24 15:04:25.671082 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:25.671050    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" path="/var/lib/kubelet/pods/9d02758d-d4c2-43e8-928a-81c69ab96cc8/volumes"
Jul 24 15:04:29.149327 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149293    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"]
Jul 24 15:04:29.149727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149672    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy"
Jul 24 15:04:29.149727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149682    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy"
Jul 24 15:04:29.149727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149700    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy"
Jul 24 15:04:29.149727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149706    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy"
Jul 24 15:04:29.149727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149715    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy"
Jul 24 15:04:29.149727 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149721    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy"
Jul 24 15:04:29.149946 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149790    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy"
Jul 24 15:04:29.149946 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.149797    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3666c6d-c419-4b20-81aa-f962c96096e1" containerName="istio-proxy"
Jul 24 15:04:29.154438 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.154414    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.157007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.156988    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0d170b-openshift-default-dockercfg-kbzp5\""
Jul 24 15:04:29.167822 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.167802    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"]
Jul 24 15:04:29.223616 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223592    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-data\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223718 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223641    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-credential-socket\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223718 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223678    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-socket\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223718 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223706    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-envoy\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223845 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223748    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-certs\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223845 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223780    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-token\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223918 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223873    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgghz\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-kube-api-access-hgghz\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223918 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223901    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-podinfo\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.223980 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.223924    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istiod-ca-cert\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.324964 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.324931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-data\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325155 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.324981    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-credential-socket\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325155 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325016    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-socket\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325155 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325041    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-envoy\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325155 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325069    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-certs\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325381 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325351    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-data\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325438 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325403    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-socket\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325438 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325415    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-token\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325433    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-credential-socket\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325504    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgghz\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-kube-api-access-hgghz\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325621 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325529    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-podinfo\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325621 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325543    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-certs\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.325621 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.325554    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istiod-ca-cert\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.326215 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.326192    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istiod-ca-cert\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.327485 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.327455    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-envoy\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.328019 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.327998    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-podinfo\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.336463 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.336440    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgghz\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-kube-api-access-hgghz\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.336657 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.336639    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-token\") pod \"e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") " pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.464741 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.464649    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:29.614097 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.614062    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"]
Jul 24 15:04:29.617518 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:04:29.617488    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b5ef37_6a59_4852_b3b2_0669c1f6f354.slice/crio-5282ef744df0a7d5c01647571a4f5f95f5b6e0aa37de960c22c7835486ef6ac2 WatchSource:0}: Error finding container 5282ef744df0a7d5c01647571a4f5f95f5b6e0aa37de960c22c7835486ef6ac2: Status 404 returned error can't find the container with id 5282ef744df0a7d5c01647571a4f5f95f5b6e0aa37de960c22c7835486ef6ac2
Jul 24 15:04:29.619691 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.619662    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:29.619747 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.619727    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:29.619788 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.619756    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:29.868823 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.868786    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" event={"ID":"16b5ef37-6a59-4852-b3b2-0669c1f6f354","Type":"ContainerStarted","Data":"65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"}
Jul 24 15:04:29.868994 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.868881    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" event={"ID":"16b5ef37-6a59-4852-b3b2-0669c1f6f354","Type":"ContainerStarted","Data":"5282ef744df0a7d5c01647571a4f5f95f5b6e0aa37de960c22c7835486ef6ac2"}
Jul 24 15:04:29.893154 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:29.893098    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" podStartSLOduration=0.893081563 podStartE2EDuration="893.081563ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:29.889416057 +0000 UTC m=+2526.674116519" watchObservedRunningTime="2026-07-24 15:04:29.893081563 +0000 UTC m=+2526.677782026"
Jul 24 15:04:30.069583 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:30.069554    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:04:30.464991 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:30.464963    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:30.469282 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:30.469259    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:30.874747 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:30.874720    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:30.875569 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:30.875552    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:37.824337 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.824244    2584 patch_prober.go:28] interesting pod/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 24 15:04:37.824337 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.824317    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 24 15:04:37.925198 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.925165    2584 generic.go:356] "Generic (PLEG): container finished" podID="1376081d-75c9-499c-b274-417f6fab91a0" containerID="f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a" exitCode=0
Jul 24 15:04:37.925371 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.925238    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" event={"ID":"1376081d-75c9-499c-b274-417f6fab91a0","Type":"ContainerDied","Data":"f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"}
Jul 24 15:04:37.925699 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.925683    2584 scope.go:117] "RemoveContainer" containerID="f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"
Jul 24 15:04:37.926735 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.926709    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:37.926812 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.926776    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:37.926812 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:37.926806    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:38.937468 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:38.937420    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" event={"ID":"1376081d-75c9-499c-b274-417f6fab91a0","Type":"ContainerStarted","Data":"72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2"}
Jul 24 15:04:39.591063 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:39.591018    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:39.591325 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:39.591297    2584 patch_prober.go:28] interesting pod/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 24 15:04:39.591404 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:39.591374    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 24 15:04:40.591798 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:40.591752    2584 patch_prober.go:28] interesting pod/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 24 15:04:40.592311 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:40.591853    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 24 15:04:41.591770 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:41.591733    2584 patch_prober.go:28] interesting pod/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 24 15:04:41.592110 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:41.592066    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 24 15:04:42.591765 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:42.591729    2584 patch_prober.go:28] interesting pod/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 24 15:04:42.592069 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:42.592039    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 24 15:04:43.596251 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:43.596221    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:43.596644 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:43.596552    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:43.597233 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:43.597215    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:04:44.971064 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:44.970975    2584 generic.go:356] "Generic (PLEG): container finished" podID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerID="65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6" exitCode=0
Jul 24 15:04:44.971064 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:44.971049    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" event={"ID":"16b5ef37-6a59-4852-b3b2-0669c1f6f354","Type":"ContainerDied","Data":"65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"}
Jul 24 15:04:44.971555 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:44.971539    2584 scope.go:117] "RemoveContainer" containerID="65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"
Jul 24 15:04:44.972469 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:44.972443    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:44.978857 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:44.972917    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:44.978857 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:44.973054    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:04:45.875432 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:45.875397    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:45.978123 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:45.978089    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" event={"ID":"16b5ef37-6a59-4852-b3b2-0669c1f6f354","Type":"ContainerStarted","Data":"018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45"}
Jul 24 15:04:46.059754 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:46.059719    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:04:46.465512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:46.465476    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:46.466730 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:46.466701    2584 patch_prober.go:28] interesting pod/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 24 15:04:46.466906 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:46.466769    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 24 15:04:47.465714 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:47.465677    2584 patch_prober.go:28] interesting pod/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 24 15:04:47.466111 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:47.465729    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 24 15:04:48.465272 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:48.465243    2584 patch_prober.go:28] interesting pod/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 24 15:04:48.465446 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:48.465293    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 24 15:04:49.465068 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:49.465033    2584 patch_prober.go:28] interesting pod/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Jul 24 15:04:49.465428 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:49.465085    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Jul 24 15:04:50.469140 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:50.469112    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:50.469577 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:50.469494    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:50.470203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:50.470183    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:04:59.854822 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:04:59.854788    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:05:42.349416 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:05:42.349386    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:06:03.854970 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:06:03.854933    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:06:21.743709 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:06:21.743666    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:06:32.958114 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:06:32.958074    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:06:52.545407 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:06:52.545372    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:07:13.144336 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:07:13.144296    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:07:23.792596 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:07:23.792519    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:07:23.797368 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:07:23.797348    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:07:30.453618 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:07:30.453585    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:07:48.550489 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:07:48.550449    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:08:00.369201 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:08:00.369158    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:08:09.956619 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:08:09.956573    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:08:24.004139 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:08:24.004109    2584 scope.go:117] "RemoveContainer" containerID="b4bb3bb3cc3e33c08bed8a35ec0594bd25fe0babd940dff1286c81ce971d5cdd"
Jul 24 15:08:36.548969 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:08:36.548936    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:08:51.652643 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:08:51.652597    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:09:13.044434 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:09:13.044396    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:09:24.024448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:09:24.024417    2584 scope.go:117] "RemoveContainer" containerID="7b547593f15a7096bd859179da2115a573d8946c7665a05c9bfab06da7153b4c"
Jul 24 15:09:24.032901 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:09:24.032881    2584 scope.go:117] "RemoveContainer" containerID="41268d9ac034721495ead31e278c1c07af05e9ca57d1a1fcc455b5c7c70247b3"
Jul 24 15:09:30.242053 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:09:30.242018    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:09:39.943200 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:09:39.943166    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:09:54.548174 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:09:54.548137    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:10:08.348324 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:08.348288    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:10:23.345932 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:23.345818    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:10:24.055324 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:24.055287    2584 scope.go:117] "RemoveContainer" containerID="4de28ddd28bca394458b5695d8d4685ad93c8cd126688c69f0cf8d610abce8b0"
Jul 24 15:10:32.621157 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.621116    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-140317205286064"]
Jul 24 15:10:32.621609 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.621598    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d02758d-d4c2-43e8-928a-81c69ab96cc8" containerName="istio-proxy"
Jul 24 15:10:32.626289 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.626263    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:32.628993 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.628967    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qp2qm\""
Jul 24 15:10:32.632671 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.632637    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205286064"]
Jul 24 15:10:32.738654 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.738613    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xt57\" (UniqueName: \"kubernetes.io/projected/5e1aef1a-064c-4fcc-bc67-90b3af20d526-kube-api-access-4xt57\") pod \"test-curl-7533-140317205286064\" (UID: \"5e1aef1a-064c-4fcc-bc67-90b3af20d526\") " pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:32.839925 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.839869    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xt57\" (UniqueName: \"kubernetes.io/projected/5e1aef1a-064c-4fcc-bc67-90b3af20d526-kube-api-access-4xt57\") pod \"test-curl-7533-140317205286064\" (UID: \"5e1aef1a-064c-4fcc-bc67-90b3af20d526\") " pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:32.850491 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.850458    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xt57\" (UniqueName: \"kubernetes.io/projected/5e1aef1a-064c-4fcc-bc67-90b3af20d526-kube-api-access-4xt57\") pod \"test-curl-7533-140317205286064\" (UID: \"5e1aef1a-064c-4fcc-bc67-90b3af20d526\") " pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:32.937570 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:32.937475    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:33.079472 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:33.079447    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205286064"]
Jul 24 15:10:33.081646 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:33.081616    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e1aef1a_064c_4fcc_bc67_90b3af20d526.slice/crio-075d98f38798e78b40e10f83227cab72b4c48dc0c4c3f3002f8607a9ae874aec WatchSource:0}: Error finding container 075d98f38798e78b40e10f83227cab72b4c48dc0c4c3f3002f8607a9ae874aec: Status 404 returned error can't find the container with id 075d98f38798e78b40e10f83227cab72b4c48dc0c4c3f3002f8607a9ae874aec
Jul 24 15:10:33.083088 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:33.083068    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:10:33.832143 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:33.832110    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205286064" event={"ID":"5e1aef1a-064c-4fcc-bc67-90b3af20d526","Type":"ContainerStarted","Data":"075d98f38798e78b40e10f83227cab72b4c48dc0c4c3f3002f8607a9ae874aec"}
Jul 24 15:10:34.838095 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:34.838059    2584 generic.go:356] "Generic (PLEG): container finished" podID="5e1aef1a-064c-4fcc-bc67-90b3af20d526" containerID="6f3f57807af7436c3ef3eae94750f3022db557aaeaa4d405c4055115bdecdcbd" exitCode=0
Jul 24 15:10:34.838505 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:34.838157    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205286064" event={"ID":"5e1aef1a-064c-4fcc-bc67-90b3af20d526","Type":"ContainerDied","Data":"6f3f57807af7436c3ef3eae94750f3022db557aaeaa4d405c4055115bdecdcbd"}
Jul 24 15:10:35.980737 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:35.980711    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:36.018496 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.018461    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7533-140317205286064_5e1aef1a-064c-4fcc-bc67-90b3af20d526/test-curl-7533-140317205286064/0.log"
Jul 24 15:10:36.045590 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.045561    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-140317205286064"]
Jul 24 15:10:36.050711 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.050681    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-140317205286064"]
Jul 24 15:10:36.170384 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.170292    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xt57\" (UniqueName: \"kubernetes.io/projected/5e1aef1a-064c-4fcc-bc67-90b3af20d526-kube-api-access-4xt57\") pod \"5e1aef1a-064c-4fcc-bc67-90b3af20d526\" (UID: \"5e1aef1a-064c-4fcc-bc67-90b3af20d526\") "
Jul 24 15:10:36.172526 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.172491    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e1aef1a-064c-4fcc-bc67-90b3af20d526-kube-api-access-4xt57" (OuterVolumeSpecName: "kube-api-access-4xt57") pod "5e1aef1a-064c-4fcc-bc67-90b3af20d526" (UID: "5e1aef1a-064c-4fcc-bc67-90b3af20d526"). InnerVolumeSpecName "kube-api-access-4xt57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:10:36.198113 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.198084    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-140317205344688"]
Jul 24 15:10:36.198511 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.198497    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e1aef1a-064c-4fcc-bc67-90b3af20d526" containerName="test-curl-7533-140317205286064"
Jul 24 15:10:36.198557 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.198513    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e1aef1a-064c-4fcc-bc67-90b3af20d526" containerName="test-curl-7533-140317205286064"
Jul 24 15:10:36.198612 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.198602    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e1aef1a-064c-4fcc-bc67-90b3af20d526" containerName="test-curl-7533-140317205286064"
Jul 24 15:10:36.201743 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.201726    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:36.208420 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.208398    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205344688"]
Jul 24 15:10:36.272013 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.271971    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xt57\" (UniqueName: \"kubernetes.io/projected/5e1aef1a-064c-4fcc-bc67-90b3af20d526-kube-api-access-4xt57\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:10:36.372924 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.372885    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6226\" (UniqueName: \"kubernetes.io/projected/6f2de8f4-875f-464f-9171-fccd97f42c62-kube-api-access-h6226\") pod \"test-curl-7533-140317205344688\" (UID: \"6f2de8f4-875f-464f-9171-fccd97f42c62\") " pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:36.473913 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.473805    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6226\" (UniqueName: \"kubernetes.io/projected/6f2de8f4-875f-464f-9171-fccd97f42c62-kube-api-access-h6226\") pod \"test-curl-7533-140317205344688\" (UID: \"6f2de8f4-875f-464f-9171-fccd97f42c62\") " pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:36.484039 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.484014    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6226\" (UniqueName: \"kubernetes.io/projected/6f2de8f4-875f-464f-9171-fccd97f42c62-kube-api-access-h6226\") pod \"test-curl-7533-140317205344688\" (UID: \"6f2de8f4-875f-464f-9171-fccd97f42c62\") " pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:36.513300 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.513272    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:36.653323 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.653189    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205344688"]
Jul 24 15:10:36.655532 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:36.655501    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f2de8f4_875f_464f_9171_fccd97f42c62.slice/crio-2dcf9ec35339a89eef80432cd8c230de76ddb1beb7e2d88d577adebb071b09ef WatchSource:0}: Error finding container 2dcf9ec35339a89eef80432cd8c230de76ddb1beb7e2d88d577adebb071b09ef: Status 404 returned error can't find the container with id 2dcf9ec35339a89eef80432cd8c230de76ddb1beb7e2d88d577adebb071b09ef
Jul 24 15:10:36.849100 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.849068    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205344688" event={"ID":"6f2de8f4-875f-464f-9171-fccd97f42c62","Type":"ContainerStarted","Data":"2dcf9ec35339a89eef80432cd8c230de76ddb1beb7e2d88d577adebb071b09ef"}
Jul 24 15:10:36.850419 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.850389    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="075d98f38798e78b40e10f83227cab72b4c48dc0c4c3f3002f8607a9ae874aec"
Jul 24 15:10:36.850419 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:36.850406    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:36.860985 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:10:36.860959    2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-140317205286064\" is forbidden: User \"system:node:ip-10-0-135-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-93.ec2.internal' and this object" podUID="5e1aef1a-064c-4fcc-bc67-90b3af20d526" pod="opendatahub/test-curl-7533-140317205286064"
Jul 24 15:10:37.671717 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:37.671684    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e1aef1a-064c-4fcc-bc67-90b3af20d526" path="/var/lib/kubelet/pods/5e1aef1a-064c-4fcc-bc67-90b3af20d526/volumes"
Jul 24 15:10:37.857430 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:37.857395    2584 generic.go:356] "Generic (PLEG): container finished" podID="6f2de8f4-875f-464f-9171-fccd97f42c62" containerID="ce2cc1705837194f532d41b9f5054140887cffd866c6696119b5ff6c65fb7b0e" exitCode=0
Jul 24 15:10:37.857615 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:37.857484    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205344688" event={"ID":"6f2de8f4-875f-464f-9171-fccd97f42c62","Type":"ContainerDied","Data":"ce2cc1705837194f532d41b9f5054140887cffd866c6696119b5ff6c65fb7b0e"}
Jul 24 15:10:38.146635 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:38.146599    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:10:39.006202 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.006180    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:39.030326 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.030302    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7533-140317205344688_6f2de8f4-875f-464f-9171-fccd97f42c62/test-curl-7533-140317205344688/0.log"
Jul 24 15:10:39.060465 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.060437    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-140317205344688"]
Jul 24 15:10:39.065509 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.065487    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-140317205344688"]
Jul 24 15:10:39.100999 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.100976    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6226\" (UniqueName: \"kubernetes.io/projected/6f2de8f4-875f-464f-9171-fccd97f42c62-kube-api-access-h6226\") pod \"6f2de8f4-875f-464f-9171-fccd97f42c62\" (UID: \"6f2de8f4-875f-464f-9171-fccd97f42c62\") "
Jul 24 15:10:39.103120 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.103097    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f2de8f4-875f-464f-9171-fccd97f42c62-kube-api-access-h6226" (OuterVolumeSpecName: "kube-api-access-h6226") pod "6f2de8f4-875f-464f-9171-fccd97f42c62" (UID: "6f2de8f4-875f-464f-9171-fccd97f42c62"). InnerVolumeSpecName "kube-api-access-h6226". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:10:39.208889 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.208736    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6226\" (UniqueName: \"kubernetes.io/projected/6f2de8f4-875f-464f-9171-fccd97f42c62-kube-api-access-h6226\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:10:39.264528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.264492    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:39.264946 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.264931    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f2de8f4-875f-464f-9171-fccd97f42c62" containerName="test-curl-7533-140317205344688"
Jul 24 15:10:39.264996 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.264947    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f2de8f4-875f-464f-9171-fccd97f42c62" containerName="test-curl-7533-140317205344688"
Jul 24 15:10:39.265029 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.265011    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f2de8f4-875f-464f-9171-fccd97f42c62" containerName="test-curl-7533-140317205344688"
Jul 24 15:10:39.268088 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.268071    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:39.275790 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.275766    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:39.309293 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.309262    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9wd8\" (UniqueName: \"kubernetes.io/projected/acb3e0e5-2c98-4cc1-bd05-90ea27f02862-kube-api-access-z9wd8\") pod \"test-curl-7533-140317204551408\" (UID: \"acb3e0e5-2c98-4cc1-bd05-90ea27f02862\") " pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:39.410556 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.410523    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9wd8\" (UniqueName: \"kubernetes.io/projected/acb3e0e5-2c98-4cc1-bd05-90ea27f02862-kube-api-access-z9wd8\") pod \"test-curl-7533-140317204551408\" (UID: \"acb3e0e5-2c98-4cc1-bd05-90ea27f02862\") " pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:39.420179 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.420151    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9wd8\" (UniqueName: \"kubernetes.io/projected/acb3e0e5-2c98-4cc1-bd05-90ea27f02862-kube-api-access-z9wd8\") pod \"test-curl-7533-140317204551408\" (UID: \"acb3e0e5-2c98-4cc1-bd05-90ea27f02862\") " pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:39.578596 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.578569    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:39.673462 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.673427    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f2de8f4-875f-464f-9171-fccd97f42c62" path="/var/lib/kubelet/pods/6f2de8f4-875f-464f-9171-fccd97f42c62/volumes"
Jul 24 15:10:39.726041 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.725994    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:39.728151 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:39.728128    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacb3e0e5_2c98_4cc1_bd05_90ea27f02862.slice/crio-d03359e2bb0a14544f467ca0f8fe56bbeb7b03d1f6805b6f2bfdff2c844e3d90 WatchSource:0}: Error finding container d03359e2bb0a14544f467ca0f8fe56bbeb7b03d1f6805b6f2bfdff2c844e3d90: Status 404 returned error can't find the container with id d03359e2bb0a14544f467ca0f8fe56bbeb7b03d1f6805b6f2bfdff2c844e3d90
Jul 24 15:10:39.868523 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.868451    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205344688"
Jul 24 15:10:39.868523 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.868501    2584 scope.go:117] "RemoveContainer" containerID="ce2cc1705837194f532d41b9f5054140887cffd866c6696119b5ff6c65fb7b0e"
Jul 24 15:10:39.869870 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:39.869772    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317204551408" event={"ID":"acb3e0e5-2c98-4cc1-bd05-90ea27f02862","Type":"ContainerStarted","Data":"d03359e2bb0a14544f467ca0f8fe56bbeb7b03d1f6805b6f2bfdff2c844e3d90"}
Jul 24 15:10:40.877281 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:40.877249    2584 generic.go:356] "Generic (PLEG): container finished" podID="acb3e0e5-2c98-4cc1-bd05-90ea27f02862" containerID="c74167c54c3a3666d3ea7d5ac9a9732ff51a2977f664eb732fe46f8bc2d857a9" exitCode=0
Jul 24 15:10:40.877696 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:40.877337    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317204551408" event={"ID":"acb3e0e5-2c98-4cc1-bd05-90ea27f02862","Type":"ContainerDied","Data":"c74167c54c3a3666d3ea7d5ac9a9732ff51a2977f664eb732fe46f8bc2d857a9"}
Jul 24 15:10:42.034601 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.034574    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.058162 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.058137    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7533-140317204551408_acb3e0e5-2c98-4cc1-bd05-90ea27f02862/test-curl-7533-140317204551408/0.log"
Jul 24 15:10:42.085381 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.085354    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:42.089978 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.089955    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:42.134326 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.134234    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9wd8\" (UniqueName: \"kubernetes.io/projected/acb3e0e5-2c98-4cc1-bd05-90ea27f02862-kube-api-access-z9wd8\") pod \"acb3e0e5-2c98-4cc1-bd05-90ea27f02862\" (UID: \"acb3e0e5-2c98-4cc1-bd05-90ea27f02862\") "
Jul 24 15:10:42.136329 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.136303    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb3e0e5-2c98-4cc1-bd05-90ea27f02862-kube-api-access-z9wd8" (OuterVolumeSpecName: "kube-api-access-z9wd8") pod "acb3e0e5-2c98-4cc1-bd05-90ea27f02862" (UID: "acb3e0e5-2c98-4cc1-bd05-90ea27f02862"). InnerVolumeSpecName "kube-api-access-z9wd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:10:42.235603 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.235570    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9wd8\" (UniqueName: \"kubernetes.io/projected/acb3e0e5-2c98-4cc1-bd05-90ea27f02862-kube-api-access-z9wd8\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:10:42.249826 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.249797    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:42.250220 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.250205    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acb3e0e5-2c98-4cc1-bd05-90ea27f02862" containerName="test-curl-7533-140317204551408"
Jul 24 15:10:42.250268 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.250221    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="acb3e0e5-2c98-4cc1-bd05-90ea27f02862" containerName="test-curl-7533-140317204551408"
Jul 24 15:10:42.250302 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.250280    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="acb3e0e5-2c98-4cc1-bd05-90ea27f02862" containerName="test-curl-7533-140317204551408"
Jul 24 15:10:42.253220 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.253206    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.258974 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.258954    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:42.336604 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.336567    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72ns8\" (UniqueName: \"kubernetes.io/projected/9908d7f4-22c0-4c80-9fff-e8c01fe530df-kube-api-access-72ns8\") pod \"test-curl-7533-140317204551408\" (UID: \"9908d7f4-22c0-4c80-9fff-e8c01fe530df\") " pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.438026 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.437925    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72ns8\" (UniqueName: \"kubernetes.io/projected/9908d7f4-22c0-4c80-9fff-e8c01fe530df-kube-api-access-72ns8\") pod \"test-curl-7533-140317204551408\" (UID: \"9908d7f4-22c0-4c80-9fff-e8c01fe530df\") " pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.447611 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.447581    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72ns8\" (UniqueName: \"kubernetes.io/projected/9908d7f4-22c0-4c80-9fff-e8c01fe530df-kube-api-access-72ns8\") pod \"test-curl-7533-140317204551408\" (UID: \"9908d7f4-22c0-4c80-9fff-e8c01fe530df\") " pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.563204 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.563172    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.699634 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.699611    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:42.701906 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:42.701879    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9908d7f4_22c0_4c80_9fff_e8c01fe530df.slice/crio-766f0bb7beccb305d6d6a2b950def93ddd85d2855b0e81ae83337d373a8ede27 WatchSource:0}: Error finding container 766f0bb7beccb305d6d6a2b950def93ddd85d2855b0e81ae83337d373a8ede27: Status 404 returned error can't find the container with id 766f0bb7beccb305d6d6a2b950def93ddd85d2855b0e81ae83337d373a8ede27
Jul 24 15:10:42.890797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.890763    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d03359e2bb0a14544f467ca0f8fe56bbeb7b03d1f6805b6f2bfdff2c844e3d90"
Jul 24 15:10:42.890797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.890775    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:42.892103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.892079    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317204551408" event={"ID":"9908d7f4-22c0-4c80-9fff-e8c01fe530df","Type":"ContainerStarted","Data":"766f0bb7beccb305d6d6a2b950def93ddd85d2855b0e81ae83337d373a8ede27"}
Jul 24 15:10:42.903458 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:42.903432    2584 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7533-140317204551408" oldPodUID="acb3e0e5-2c98-4cc1-bd05-90ea27f02862" podUID="9908d7f4-22c0-4c80-9fff-e8c01fe530df"
Jul 24 15:10:43.672250 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:43.672210    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acb3e0e5-2c98-4cc1-bd05-90ea27f02862" path="/var/lib/kubelet/pods/acb3e0e5-2c98-4cc1-bd05-90ea27f02862/volumes"
Jul 24 15:10:43.900803 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:43.900767    2584 generic.go:356] "Generic (PLEG): container finished" podID="9908d7f4-22c0-4c80-9fff-e8c01fe530df" containerID="16b298d41ace13073636e72125bd7b1a5cf28b943cb88ac2ba91a1f620ab0fd2" exitCode=0
Jul 24 15:10:43.901049 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:43.900817    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317204551408" event={"ID":"9908d7f4-22c0-4c80-9fff-e8c01fe530df","Type":"ContainerDied","Data":"16b298d41ace13073636e72125bd7b1a5cf28b943cb88ac2ba91a1f620ab0fd2"}
Jul 24 15:10:45.033926 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.033901    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:45.057372 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.057349    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7533-140317204551408_9908d7f4-22c0-4c80-9fff-e8c01fe530df/test-curl-7533-140317204551408/0.log"
Jul 24 15:10:45.060641 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.060621    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72ns8\" (UniqueName: \"kubernetes.io/projected/9908d7f4-22c0-4c80-9fff-e8c01fe530df-kube-api-access-72ns8\") pod \"9908d7f4-22c0-4c80-9fff-e8c01fe530df\" (UID: \"9908d7f4-22c0-4c80-9fff-e8c01fe530df\") "
Jul 24 15:10:45.062672 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.062649    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9908d7f4-22c0-4c80-9fff-e8c01fe530df-kube-api-access-72ns8" (OuterVolumeSpecName: "kube-api-access-72ns8") pod "9908d7f4-22c0-4c80-9fff-e8c01fe530df" (UID: "9908d7f4-22c0-4c80-9fff-e8c01fe530df"). InnerVolumeSpecName "kube-api-access-72ns8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:10:45.085010 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.084979    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:45.090213 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.090186    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-140317204551408"]
Jul 24 15:10:45.162409 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.162315    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72ns8\" (UniqueName: \"kubernetes.io/projected/9908d7f4-22c0-4c80-9fff-e8c01fe530df-kube-api-access-72ns8\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:10:45.233597 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.233564    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:45.234016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.233985    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9908d7f4-22c0-4c80-9fff-e8c01fe530df" containerName="test-curl-7533-140317204551408"
Jul 24 15:10:45.234071 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.234018    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9908d7f4-22c0-4c80-9fff-e8c01fe530df" containerName="test-curl-7533-140317204551408"
Jul 24 15:10:45.234109 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.234084    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9908d7f4-22c0-4c80-9fff-e8c01fe530df" containerName="test-curl-7533-140317204551408"
Jul 24 15:10:45.237086 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.237070    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:45.244246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.244218    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:45.263528 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.263497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmhf\" (UniqueName: \"kubernetes.io/projected/ead900d5-09a5-4e49-b65a-ac4ba6520a48-kube-api-access-cbmhf\") pod \"test-curl-7533-140317205448144\" (UID: \"ead900d5-09a5-4e49-b65a-ac4ba6520a48\") " pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:45.364520 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.364488    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmhf\" (UniqueName: \"kubernetes.io/projected/ead900d5-09a5-4e49-b65a-ac4ba6520a48-kube-api-access-cbmhf\") pod \"test-curl-7533-140317205448144\" (UID: \"ead900d5-09a5-4e49-b65a-ac4ba6520a48\") " pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:45.375883 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.375856    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmhf\" (UniqueName: \"kubernetes.io/projected/ead900d5-09a5-4e49-b65a-ac4ba6520a48-kube-api-access-cbmhf\") pod \"test-curl-7533-140317205448144\" (UID: \"ead900d5-09a5-4e49-b65a-ac4ba6520a48\") " pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:45.547996 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.547969    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:45.670990 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.670961    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9908d7f4-22c0-4c80-9fff-e8c01fe530df" path="/var/lib/kubelet/pods/9908d7f4-22c0-4c80-9fff-e8c01fe530df/volumes"
Jul 24 15:10:45.685547 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.684865    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:45.687354 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:45.687329    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podead900d5_09a5_4e49_b65a_ac4ba6520a48.slice/crio-af2323a798f09c9e16fa7e28ef70c99f31c09603febd073d762ccade2489e2e3 WatchSource:0}: Error finding container af2323a798f09c9e16fa7e28ef70c99f31c09603febd073d762ccade2489e2e3: Status 404 returned error can't find the container with id af2323a798f09c9e16fa7e28ef70c99f31c09603febd073d762ccade2489e2e3
Jul 24 15:10:45.912472 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.912379    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205448144" event={"ID":"ead900d5-09a5-4e49-b65a-ac4ba6520a48","Type":"ContainerStarted","Data":"af2323a798f09c9e16fa7e28ef70c99f31c09603febd073d762ccade2489e2e3"}
Jul 24 15:10:45.913561 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.913543    2584 scope.go:117] "RemoveContainer" containerID="16b298d41ace13073636e72125bd7b1a5cf28b943cb88ac2ba91a1f620ab0fd2"
Jul 24 15:10:45.913679 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:45.913566    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317204551408"
Jul 24 15:10:46.920536 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:46.920503    2584 generic.go:356] "Generic (PLEG): container finished" podID="ead900d5-09a5-4e49-b65a-ac4ba6520a48" containerID="fbff283c238b1b2a30cc022b1f381793d5142ceb69f2cb38bc07c725df4b70cf" exitCode=0
Jul 24 15:10:46.920971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:46.920588    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205448144" event={"ID":"ead900d5-09a5-4e49-b65a-ac4ba6520a48","Type":"ContainerDied","Data":"fbff283c238b1b2a30cc022b1f381793d5142ceb69f2cb38bc07c725df4b70cf"}
Jul 24 15:10:48.071725 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.071703    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.093045 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.093005    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7533-140317205448144_ead900d5-09a5-4e49-b65a-ac4ba6520a48/test-curl-7533-140317205448144/0.log"
Jul 24 15:10:48.119007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.118985    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:48.122123 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.122100    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:48.192712 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.192621    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbmhf\" (UniqueName: \"kubernetes.io/projected/ead900d5-09a5-4e49-b65a-ac4ba6520a48-kube-api-access-cbmhf\") pod \"ead900d5-09a5-4e49-b65a-ac4ba6520a48\" (UID: \"ead900d5-09a5-4e49-b65a-ac4ba6520a48\") "
Jul 24 15:10:48.194846 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.194800    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead900d5-09a5-4e49-b65a-ac4ba6520a48-kube-api-access-cbmhf" (OuterVolumeSpecName: "kube-api-access-cbmhf") pod "ead900d5-09a5-4e49-b65a-ac4ba6520a48" (UID: "ead900d5-09a5-4e49-b65a-ac4ba6520a48"). InnerVolumeSpecName "kube-api-access-cbmhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:10:48.292548 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.292513    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:48.292973 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.292958    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ead900d5-09a5-4e49-b65a-ac4ba6520a48" containerName="test-curl-7533-140317205448144"
Jul 24 15:10:48.293042 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.292975    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ead900d5-09a5-4e49-b65a-ac4ba6520a48" containerName="test-curl-7533-140317205448144"
Jul 24 15:10:48.293082 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.293046    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ead900d5-09a5-4e49-b65a-ac4ba6520a48" containerName="test-curl-7533-140317205448144"
Jul 24 15:10:48.293765 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.293739    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmhf\" (UniqueName: \"kubernetes.io/projected/ead900d5-09a5-4e49-b65a-ac4ba6520a48-kube-api-access-cbmhf\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:10:48.296254 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.296237    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.302532 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.302513    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:48.394524 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.394496    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlj8\" (UniqueName: \"kubernetes.io/projected/f2be133f-e39e-4eb0-adee-c0dc25529e25-kube-api-access-2nlj8\") pod \"test-curl-7533-140317205448144\" (UID: \"f2be133f-e39e-4eb0-adee-c0dc25529e25\") " pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.495636 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.495552    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlj8\" (UniqueName: \"kubernetes.io/projected/f2be133f-e39e-4eb0-adee-c0dc25529e25-kube-api-access-2nlj8\") pod \"test-curl-7533-140317205448144\" (UID: \"f2be133f-e39e-4eb0-adee-c0dc25529e25\") " pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.505797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.505767    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlj8\" (UniqueName: \"kubernetes.io/projected/f2be133f-e39e-4eb0-adee-c0dc25529e25-kube-api-access-2nlj8\") pod \"test-curl-7533-140317205448144\" (UID: \"f2be133f-e39e-4eb0-adee-c0dc25529e25\") " pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.605884 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.605853    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.739946 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.739920    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:48.742435 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:48.742407    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2be133f_e39e_4eb0_adee_c0dc25529e25.slice/crio-718060ffad9143b1da5380d1d6bb96e1ca3ef8a2a70a368d1093472ab32c8038 WatchSource:0}: Error finding container 718060ffad9143b1da5380d1d6bb96e1ca3ef8a2a70a368d1093472ab32c8038: Status 404 returned error can't find the container with id 718060ffad9143b1da5380d1d6bb96e1ca3ef8a2a70a368d1093472ab32c8038
Jul 24 15:10:48.934055 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.934028    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:48.934230 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.934028    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af2323a798f09c9e16fa7e28ef70c99f31c09603febd073d762ccade2489e2e3"
Jul 24 15:10:48.935310 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.935286    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205448144" event={"ID":"f2be133f-e39e-4eb0-adee-c0dc25529e25","Type":"ContainerStarted","Data":"718060ffad9143b1da5380d1d6bb96e1ca3ef8a2a70a368d1093472ab32c8038"}
Jul 24 15:10:48.946778 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:48.946753    2584 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7533-140317205448144" oldPodUID="ead900d5-09a5-4e49-b65a-ac4ba6520a48" podUID="f2be133f-e39e-4eb0-adee-c0dc25529e25"
Jul 24 15:10:49.671043 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:49.671009    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ead900d5-09a5-4e49-b65a-ac4ba6520a48" path="/var/lib/kubelet/pods/ead900d5-09a5-4e49-b65a-ac4ba6520a48/volumes"
Jul 24 15:10:49.941448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:49.941362    2584 generic.go:356] "Generic (PLEG): container finished" podID="f2be133f-e39e-4eb0-adee-c0dc25529e25" containerID="698f3961ec569e77556021552290ffc16a7e2d77c5e50c6462a378aa81cedd5a" exitCode=0
Jul 24 15:10:49.941590 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:49.941452    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-140317205448144" event={"ID":"f2be133f-e39e-4eb0-adee-c0dc25529e25","Type":"ContainerDied","Data":"698f3961ec569e77556021552290ffc16a7e2d77c5e50c6462a378aa81cedd5a"}
Jul 24 15:10:51.094671 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.094644    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:51.119655 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.119623    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7533-140317205448144_f2be133f-e39e-4eb0-adee-c0dc25529e25/test-curl-7533-140317205448144/0.log"
Jul 24 15:10:51.149393 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.149364    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:51.154428 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.154405    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-140317205448144"]
Jul 24 15:10:51.224275 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.224181    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nlj8\" (UniqueName: \"kubernetes.io/projected/f2be133f-e39e-4eb0-adee-c0dc25529e25-kube-api-access-2nlj8\") pod \"f2be133f-e39e-4eb0-adee-c0dc25529e25\" (UID: \"f2be133f-e39e-4eb0-adee-c0dc25529e25\") "
Jul 24 15:10:51.226334 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.226304    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2be133f-e39e-4eb0-adee-c0dc25529e25-kube-api-access-2nlj8" (OuterVolumeSpecName: "kube-api-access-2nlj8") pod "f2be133f-e39e-4eb0-adee-c0dc25529e25" (UID: "f2be133f-e39e-4eb0-adee-c0dc25529e25"). InnerVolumeSpecName "kube-api-access-2nlj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:10:51.325747 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.325712    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nlj8\" (UniqueName: \"kubernetes.io/projected/f2be133f-e39e-4eb0-adee-c0dc25529e25-kube-api-access-2nlj8\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:10:51.671263 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.671232    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2be133f-e39e-4eb0-adee-c0dc25529e25" path="/var/lib/kubelet/pods/f2be133f-e39e-4eb0-adee-c0dc25529e25/volumes"
Jul 24 15:10:51.960809 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.960721    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-140317205448144"
Jul 24 15:10:51.960809 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:51.960762    2584 scope.go:117] "RemoveContainer" containerID="698f3961ec569e77556021552290ffc16a7e2d77c5e50c6462a378aa81cedd5a"
Jul 24 15:10:52.197987 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.197945    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"]
Jul 24 15:10:52.198585 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.198566    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2be133f-e39e-4eb0-adee-c0dc25529e25" containerName="test-curl-7533-140317205448144"
Jul 24 15:10:52.198625 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.198587    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2be133f-e39e-4eb0-adee-c0dc25529e25" containerName="test-curl-7533-140317205448144"
Jul 24 15:10:52.198734 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.198722    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2be133f-e39e-4eb0-adee-c0dc25529e25" containerName="test-curl-7533-140317205448144"
Jul 24 15:10:52.203677 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.203653    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.207508 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.207485    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-600d4a-openshift-default-dockercfg-vb96h\""
Jul 24 15:10:52.218600 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.218522    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"]
Jul 24 15:10:52.337538 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337497    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-podinfo\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.337538 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337534    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-token\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.337794 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337557    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-socket\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.337794 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337608    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-credential-socket\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.337961 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-data\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.337961 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337911    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istiod-ca-cert\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.338068 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.337978    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-envoy\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.338068 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.338015    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrgww\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-kube-api-access-xrgww\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.338169 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.338094    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-certs\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439047 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439008    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istiod-ca-cert\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439085    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-envoy\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439120    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrgww\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-kube-api-access-xrgww\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439157    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-certs\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439181    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-podinfo\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439246 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439205    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-token\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439541 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439384    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-socket\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439541 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439447    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-credential-socket\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439653 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439597    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-certs\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439653 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439605    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-data\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439800 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439775    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-socket\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.439887 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.439861    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istiod-ca-cert\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.440087 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.440063    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-data\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.440206 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.440096    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-credential-socket\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.441620 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.441601    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-envoy\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.442031 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.442012    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-podinfo\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.449194 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.449171    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrgww\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-kube-api-access-xrgww\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.449301 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.449199    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-token\") pod \"e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") " pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.514705 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.514612    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:52.696577 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.696555    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"]
Jul 24 15:10:52.698569 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:10:52.698529    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c7e964_a5d9_4d47_aa27_d9ef454723a6.slice/crio-0c806d7dc54dc028ba44ab885aaa0b9e6433bad355ea120aecff01e126bd1784 WatchSource:0}: Error finding container 0c806d7dc54dc028ba44ab885aaa0b9e6433bad355ea120aecff01e126bd1784: Status 404 returned error can't find the container with id 0c806d7dc54dc028ba44ab885aaa0b9e6433bad355ea120aecff01e126bd1784
Jul 24 15:10:52.701192 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.701151    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:10:52.701314 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.701241    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:10:52.701314 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.701283    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:10:52.968435 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.968399    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" event={"ID":"57c7e964-a5d9-4d47-aa27-d9ef454723a6","Type":"ContainerStarted","Data":"d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"}
Jul 24 15:10:52.968617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.968451    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" event={"ID":"57c7e964-a5d9-4d47-aa27-d9ef454723a6","Type":"ContainerStarted","Data":"0c806d7dc54dc028ba44ab885aaa0b9e6433bad355ea120aecff01e126bd1784"}
Jul 24 15:10:52.990794 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:52.990744    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podStartSLOduration=0.990728186 podStartE2EDuration="990.728186ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:10:52.98914284 +0000 UTC m=+2909.773843307" watchObservedRunningTime="2026-07-24 15:10:52.990728186 +0000 UTC m=+2909.775428650"
Jul 24 15:10:53.514878 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:53.514809    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:53.516447 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:53.516416    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 24 15:10:53.516566 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:53.516464    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 24 15:10:54.519972 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:54.519942    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:54.979750 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:54.979663    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:10:54.980782 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:10:54.980764    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:11:03.249008 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:03.248930    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:11:11.021017 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.020978    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 15:11:11.021479 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.021050    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 15:11:11.243289 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.243253    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"]
Jul 24 15:11:11.252693 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.252659    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.255856 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.255810    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-5c17a6-openshift-default-dockercfg-jmsdm\""
Jul 24 15:11:11.256817 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.256794    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"]
Jul 24 15:11:11.338577 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338540    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-certs\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338595    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-envoy\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338634    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-data\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338663    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r226h\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-kube-api-access-r226h\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338713    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5080429-445a-4b09-be98-693ff0cabe1f-istio-podinfo\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338758    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-istio-token\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338878    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-credential-socket\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.338977    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-socket\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.340758 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.339018    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5080429-445a-4b09-be98-693ff0cabe1f-istiod-ca-cert\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440289 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440260    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-istio-token\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440299    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-credential-socket\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440452 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440421    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-socket\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440565 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440477    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5080429-445a-4b09-be98-693ff0cabe1f-istiod-ca-cert\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440565 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440549    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-certs\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440574    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-envoy\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440608    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-data\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440627    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r226h\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-kube-api-access-r226h\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440666    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-credential-socket\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440906 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440671    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5080429-445a-4b09-be98-693ff0cabe1f-istio-podinfo\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.440906 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440772    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-socket\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.441008 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.440951    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-certs\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.441132 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.441106    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-data\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.441292 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.441271    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5080429-445a-4b09-be98-693ff0cabe1f-istiod-ca-cert\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.442844 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.442814    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-envoy\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.442942 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.442926    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5080429-445a-4b09-be98-693ff0cabe1f-istio-podinfo\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.449034 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.449012    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-istio-token\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.449162 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.449141    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r226h\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-kube-api-access-r226h\") pod \"e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") " pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.566674 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.566591    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:11.708846 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.708804    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"]
Jul 24 15:11:11.710702 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:11:11.710678    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5080429_445a_4b09_be98_693ff0cabe1f.slice/crio-9f2a4a84b729d6b9f86b0083b88eb3f536683f5edf9ae6e4b03896843e062daa WatchSource:0}: Error finding container 9f2a4a84b729d6b9f86b0083b88eb3f536683f5edf9ae6e4b03896843e062daa: Status 404 returned error can't find the container with id 9f2a4a84b729d6b9f86b0083b88eb3f536683f5edf9ae6e4b03896843e062daa
Jul 24 15:11:11.713432 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.713392    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:11.713525 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.713476    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:11.713525 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:11.713514    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:12.073884 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.073823    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" event={"ID":"a5080429-445a-4b09-be98-693ff0cabe1f","Type":"ContainerStarted","Data":"0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"}
Jul 24 15:11:12.074363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.073915    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" event={"ID":"a5080429-445a-4b09-be98-693ff0cabe1f","Type":"ContainerStarted","Data":"9f2a4a84b729d6b9f86b0083b88eb3f536683f5edf9ae6e4b03896843e062daa"}
Jul 24 15:11:12.075670 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.075642    2584 generic.go:356] "Generic (PLEG): container finished" podID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerID="d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369" exitCode=0
Jul 24 15:11:12.075809 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.075716    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" event={"ID":"57c7e964-a5d9-4d47-aa27-d9ef454723a6","Type":"ContainerDied","Data":"d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"}
Jul 24 15:11:12.076223 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.076205    2584 scope.go:117] "RemoveContainer" containerID="d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"
Jul 24 15:11:12.077408 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.077375    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:12.077513 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.077450    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:12.077513 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.077479    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:12.096235 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.096177    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podStartSLOduration=1.096152511 podStartE2EDuration="1.096152511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:11:12.094764837 +0000 UTC m=+2928.879465313" watchObservedRunningTime="2026-07-24 15:11:12.096152511 +0000 UTC m=+2928.880852977"
Jul 24 15:11:12.567802 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.567763    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:12.573779 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:12.573753    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:13.083119 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:13.083083    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" event={"ID":"57c7e964-a5d9-4d47-aa27-d9ef454723a6","Type":"ContainerStarted","Data":"64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee"}
Jul 24 15:11:13.083618 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:13.083522    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:13.084682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:13.084653    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:13.515383 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:13.515292    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:11:13.515581 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:13.515557    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 24 15:11:13.515626 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:13.515611    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 24 15:11:14.515354 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:14.515315    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 24 15:11:14.515865 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:14.515392    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 24 15:11:15.515740 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:15.515700    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 24 15:11:15.516203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:15.515769    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 24 15:11:16.515339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:16.515297    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Jul 24 15:11:16.515543 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:16.515376    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Jul 24 15:11:17.519286 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:17.519250    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:11:17.519807 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:17.519639    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:11:17.520512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:17.520492    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:11:21.374497 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:21.374455    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:11:29.126041 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:29.125998    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 15:11:29.126555 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:29.126075    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 15:11:30.201251 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:30.201150    2584 generic.go:356] "Generic (PLEG): container finished" podID="a5080429-445a-4b09-be98-693ff0cabe1f" containerID="0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755" exitCode=0
Jul 24 15:11:30.201251 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:30.201229    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" event={"ID":"a5080429-445a-4b09-be98-693ff0cabe1f","Type":"ContainerDied","Data":"0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"}
Jul 24 15:11:30.201715 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:30.201656    2584 scope.go:117] "RemoveContainer" containerID="0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"
Jul 24 15:11:30.202814 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:30.202786    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:30.202961 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:30.202893    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:30.202961 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:30.202945    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:11:31.207791 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:31.207757    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" event={"ID":"a5080429-445a-4b09-be98-693ff0cabe1f","Type":"ContainerStarted","Data":"ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207"}
Jul 24 15:11:31.567736 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:31.567701    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:31.567959 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:31.567875    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 24 15:11:31.567959 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:31.567935    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 24 15:11:32.567775 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:32.567725    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 24 15:11:32.568306 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:32.567792    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 24 15:11:33.567410 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:33.567371    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 24 15:11:33.567635 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:33.567427    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 24 15:11:34.567586 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:34.567555    2584 patch_prober.go:28] interesting pod/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 24 15:11:34.567997 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:34.567610    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 24 15:11:35.570595 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:35.570567    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:35.571077 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:35.571052    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:35.571704 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:35.571686    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:11:48.355662 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:11:48.355578    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:12:04.467047 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:04.467011    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:12:16.688577 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.688539    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"]
Jul 24 15:12:16.693408 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.693386    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.695871 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.695846    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-891ff3-openshift-default-dockercfg-gvncx\""
Jul 24 15:12:16.704729 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.704706    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"]
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789467    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-socket\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789593    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-certs\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789671    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789707    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-envoy\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-data\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789788    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-credential-socket\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789812    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-token\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.789860 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789856    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnphx\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-kube-api-access-fnphx\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.790476 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.789883    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-podinfo\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.890943 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.890900    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-envoy\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891147 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.890961    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-data\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891147 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.890996    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-credential-socket\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891147 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891019    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-token\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891147 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891044    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnphx\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-kube-api-access-fnphx\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891147 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891070    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-podinfo\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891147 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891141    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-socket\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-certs\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891272    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891472    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-credential-socket\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891642 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891610    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-data\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.891883 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891859    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-socket\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.892051 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.891996    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istiod-ca-cert\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.892435 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.892411    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-certs\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.894002 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.893973    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-envoy\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.894132 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.894111    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-podinfo\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.904619 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.904594    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnphx\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-kube-api-access-fnphx\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:16.906781 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:16.906760    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-token\") pod \"e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") " pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:17.006706 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.006610    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:17.171344 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.171319    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"]
Jul 24 15:12:17.173078 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:12:17.173052    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded59631e_d4f0_4c7e_ae80_b162164d9aa2.slice/crio-94a45b5943386a0dd738ccd22f4682505b314f419eeb5b6941d327d2ac6623de WatchSource:0}: Error finding container 94a45b5943386a0dd738ccd22f4682505b314f419eeb5b6941d327d2ac6623de: Status 404 returned error can't find the container with id 94a45b5943386a0dd738ccd22f4682505b314f419eeb5b6941d327d2ac6623de
Jul 24 15:12:17.175436 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.175406    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:12:17.175512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.175475    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:12:17.175512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.175500    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:12:17.471321 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.471287    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" event={"ID":"ed59631e-d4f0-4c7e-ae80-b162164d9aa2","Type":"ContainerStarted","Data":"11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"}
Jul 24 15:12:17.471481 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.471337    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" event={"ID":"ed59631e-d4f0-4c7e-ae80-b162164d9aa2","Type":"ContainerStarted","Data":"94a45b5943386a0dd738ccd22f4682505b314f419eeb5b6941d327d2ac6623de"}
Jul 24 15:12:17.493506 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:17.493443    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podStartSLOduration=1.493422336 podStartE2EDuration="1.493422336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:12:17.49304332 +0000 UTC m=+2994.277743785" watchObservedRunningTime="2026-07-24 15:12:17.493422336 +0000 UTC m=+2994.278122802"
Jul 24 15:12:18.007101 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:18.007065    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:18.012863 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:18.012815    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:18.476695 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:18.476662    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:18.477512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:18.477495    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:23.826294 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:23.826266    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:12:23.832476 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:23.832457    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:12:25.053337 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:25.053305    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:12:33.478285 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.478188    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 24 15:12:33.478285 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.478262    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 24 15:12:33.565778 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.565750    2584 generic.go:356] "Generic (PLEG): container finished" podID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerID="11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20" exitCode=0
Jul 24 15:12:33.565949 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.565823    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" event={"ID":"ed59631e-d4f0-4c7e-ae80-b162164d9aa2","Type":"ContainerDied","Data":"11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"}
Jul 24 15:12:33.566233 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.566215    2584 scope.go:117] "RemoveContainer" containerID="11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"
Jul 24 15:12:33.567254 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.567230    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:12:33.567314 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.567294    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:12:33.567351 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:33.567323    2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 15:12:34.573327 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:34.573289    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" event={"ID":"ed59631e-d4f0-4c7e-ae80-b162164d9aa2","Type":"ContainerStarted","Data":"7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f"}
Jul 24 15:12:35.007362 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.007190    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:35.007362 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.007259    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 24 15:12:35.007362 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.007303    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 24 15:12:35.133846 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.133809    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"]
Jul 24 15:12:35.137361 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.137341    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:35.139733 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.139713    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:12:35.139827 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.139750    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:12:35.145333 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.145309    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"]
Jul 24 15:12:35.172153 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.172127    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ad19c46-70e4-448e-8b6f-3979542a103a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5\" (UID: \"2ad19c46-70e4-448e-8b6f-3979542a103a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:35.273186 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.273155    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ad19c46-70e4-448e-8b6f-3979542a103a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5\" (UID: \"2ad19c46-70e4-448e-8b6f-3979542a103a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:35.273761 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.273742    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ad19c46-70e4-448e-8b6f-3979542a103a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5\" (UID: \"2ad19c46-70e4-448e-8b6f-3979542a103a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:35.448984 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.448953    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:35.644269 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:35.644244    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"]
Jul 24 15:12:35.645919 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:12:35.645890    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad19c46_70e4_448e_8b6f_3979542a103a.slice/crio-b0e13b0a2df6ab57658f2379bb1f478daddccf2b8bf78ad21675cc44ca739b53 WatchSource:0}: Error finding container b0e13b0a2df6ab57658f2379bb1f478daddccf2b8bf78ad21675cc44ca739b53: Status 404 returned error can't find the container with id b0e13b0a2df6ab57658f2379bb1f478daddccf2b8bf78ad21675cc44ca739b53
Jul 24 15:12:36.007943 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:36.007897    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 24 15:12:36.008138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:36.007965    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 24 15:12:36.589275 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:36.589215    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5" event={"ID":"2ad19c46-70e4-448e-8b6f-3979542a103a","Type":"ContainerStarted","Data":"b0ea3016f11d3db16ff44d2e36c07e91ec72d6e2cc26f494e973f20637568362"}
Jul 24 15:12:36.589482 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:36.589300    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5" event={"ID":"2ad19c46-70e4-448e-8b6f-3979542a103a","Type":"ContainerStarted","Data":"b0e13b0a2df6ab57658f2379bb1f478daddccf2b8bf78ad21675cc44ca739b53"}
Jul 24 15:12:37.007330 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:37.007236    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 24 15:12:37.008285 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:37.007895    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 24 15:12:37.599785 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:37.599744    2584 generic.go:356] "Generic (PLEG): container finished" podID="2ad19c46-70e4-448e-8b6f-3979542a103a" containerID="b0ea3016f11d3db16ff44d2e36c07e91ec72d6e2cc26f494e973f20637568362" exitCode=0
Jul 24 15:12:37.599986 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:37.599950    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5" event={"ID":"2ad19c46-70e4-448e-8b6f-3979542a103a","Type":"ContainerDied","Data":"b0ea3016f11d3db16ff44d2e36c07e91ec72d6e2cc26f494e973f20637568362"}
Jul 24 15:12:38.007140 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:38.007041    2584 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Jul 24 15:12:38.007417 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:38.007372    2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Jul 24 15:12:38.738602 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:38.738580    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:38.810979 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:38.810948    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ad19c46-70e4-448e-8b6f-3979542a103a-maas-api-service-ca\") pod \"2ad19c46-70e4-448e-8b6f-3979542a103a\" (UID: \"2ad19c46-70e4-448e-8b6f-3979542a103a\") "
Jul 24 15:12:38.811367 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:38.811343    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ad19c46-70e4-448e-8b6f-3979542a103a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ad19c46-70e4-448e-8b6f-3979542a103a" (UID: "2ad19c46-70e4-448e-8b6f-3979542a103a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:12:38.912093 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:38.912009    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ad19c46-70e4-448e-8b6f-3979542a103a-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:12:39.011311 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:39.011281    2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:39.011668 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:39.011650    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:39.012429 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:39.012409    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:12:39.612103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:39.612077    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"
Jul 24 15:12:39.612275 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:39.612112    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5" event={"ID":"2ad19c46-70e4-448e-8b6f-3979542a103a","Type":"ContainerDied","Data":"b0e13b0a2df6ab57658f2379bb1f478daddccf2b8bf78ad21675cc44ca739b53"}
Jul 24 15:12:39.612275 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:39.612175    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0e13b0a2df6ab57658f2379bb1f478daddccf2b8bf78ad21675cc44ca739b53"
Jul 24 15:12:50.226523 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:50.226486    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"]
Jul 24 15:12:50.229821 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:50.229799    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-891ff3-mxnc5"]
Jul 24 15:12:50.748306 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:50.748268    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:12:51.672657 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:51.672627    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad19c46-70e4-448e-8b6f-3979542a103a" path="/var/lib/kubelet/pods/2ad19c46-70e4-448e-8b6f-3979542a103a/volumes"
Jul 24 15:12:59.056596 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:59.056556    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"]
Jul 24 15:12:59.057103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:12:59.056766    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy" containerID="cri-o://7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f" gracePeriod=30
Jul 24 15:13:03.133149 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.133071    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"]
Jul 24 15:13:03.133542 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.133486    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad19c46-70e4-448e-8b6f-3979542a103a" containerName="revoke-keys"
Jul 24 15:13:03.133542 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.133499    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad19c46-70e4-448e-8b6f-3979542a103a" containerName="revoke-keys"
Jul 24 15:13:03.133614 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.133576    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad19c46-70e4-448e-8b6f-3979542a103a" containerName="revoke-keys"
Jul 24 15:13:03.138005 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.137988    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:03.140461 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.140436    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:13:03.141218 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.141196    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:13:03.143557 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.143533    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"]
Jul 24 15:13:03.243803 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.243777    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e27d422-d780-40e2-99bf-f75797fd8924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v\" (UID: \"4e27d422-d780-40e2-99bf-f75797fd8924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:03.344850 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.344802    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e27d422-d780-40e2-99bf-f75797fd8924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v\" (UID: \"4e27d422-d780-40e2-99bf-f75797fd8924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:03.345458 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.345432    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e27d422-d780-40e2-99bf-f75797fd8924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v\" (UID: \"4e27d422-d780-40e2-99bf-f75797fd8924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:03.450448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.450393    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:03.588667 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.588636    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"]
Jul 24 15:13:03.590751 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:13:03.590722    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e27d422_d780_40e2_99bf_f75797fd8924.slice/crio-8bf2ba0d0baa707e3e57f64dd7357d51acdd6fa09f36e6597e6d642c88b50afb WatchSource:0}: Error finding container 8bf2ba0d0baa707e3e57f64dd7357d51acdd6fa09f36e6597e6d642c88b50afb: Status 404 returned error can't find the container with id 8bf2ba0d0baa707e3e57f64dd7357d51acdd6fa09f36e6597e6d642c88b50afb
Jul 24 15:13:03.744788 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:03.744705    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v" event={"ID":"4e27d422-d780-40e2-99bf-f75797fd8924","Type":"ContainerStarted","Data":"8bf2ba0d0baa707e3e57f64dd7357d51acdd6fa09f36e6597e6d642c88b50afb"}
Jul 24 15:13:04.301369 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.301346    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:13:04.455295 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455267    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-credential-socket\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455295 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455302    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-data\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455525 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455333    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnphx\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-kube-api-access-fnphx\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455525 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455381    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-certs\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455525 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455472    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-token\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455525 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455508    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-podinfo\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455716 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455531    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istiod-ca-cert\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455716 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455557    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-envoy\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455716 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455564    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:13:04.455716 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455586    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-socket\") pod \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\" (UID: \"ed59631e-d4f0-4c7e-ae80-b162164d9aa2\") "
Jul 24 15:13:04.455956 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455802    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:13:04.455956 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.455820    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:13:04.456092 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.456076    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.456151 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.456097    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.456151 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.456111    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.456151 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.456101    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:13:04.456306 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.456173    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:13:04.457704 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.457678    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-kube-api-access-fnphx" (OuterVolumeSpecName: "kube-api-access-fnphx") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "kube-api-access-fnphx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:13:04.457704 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.457696    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:13:04.458144 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.458126    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:13:04.458309 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.458280    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed59631e-d4f0-4c7e-ae80-b162164d9aa2" (UID: "ed59631e-d4f0-4c7e-ae80-b162164d9aa2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:13:04.557363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.557328    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.557363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.557357    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.557363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.557366    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.557363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.557373    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.557623 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.557383    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.557623 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.557390    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnphx\" (UniqueName: \"kubernetes.io/projected/ed59631e-d4f0-4c7e-ae80-b162164d9aa2-kube-api-access-fnphx\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:04.751773 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.751691    2584 generic.go:356] "Generic (PLEG): container finished" podID="4e27d422-d780-40e2-99bf-f75797fd8924" containerID="b2fb9dff8b086812b19dd7813766e05daf089288a41503bab47c6c3c4e5d26f7" exitCode=0
Jul 24 15:13:04.751979 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.751776    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v" event={"ID":"4e27d422-d780-40e2-99bf-f75797fd8924","Type":"ContainerDied","Data":"b2fb9dff8b086812b19dd7813766e05daf089288a41503bab47c6c3c4e5d26f7"}
Jul 24 15:13:04.753468 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.753442    2584 generic.go:356] "Generic (PLEG): container finished" podID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerID="7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f" exitCode=0
Jul 24 15:13:04.753617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.753488    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"
Jul 24 15:13:04.753617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.753529    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" event={"ID":"ed59631e-d4f0-4c7e-ae80-b162164d9aa2","Type":"ContainerDied","Data":"7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f"}
Jul 24 15:13:04.753617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.753587    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf" event={"ID":"ed59631e-d4f0-4c7e-ae80-b162164d9aa2","Type":"ContainerDied","Data":"94a45b5943386a0dd738ccd22f4682505b314f419eeb5b6941d327d2ac6623de"}
Jul 24 15:13:04.753775 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.753618    2584 scope.go:117] "RemoveContainer" containerID="7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f"
Jul 24 15:13:04.769057 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.769039    2584 scope.go:117] "RemoveContainer" containerID="11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"
Jul 24 15:13:04.777489 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.777472    2584 scope.go:117] "RemoveContainer" containerID="7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f"
Jul 24 15:13:04.777719 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:13:04.777704    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f\": container with ID starting with 7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f not found: ID does not exist" containerID="7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f"
Jul 24 15:13:04.777765 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.777726    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f"} err="failed to get container status \"7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f\": rpc error: code = NotFound desc = could not find container \"7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f\": container with ID starting with 7fbbee6151c4131b77768eb58828f5f8b292c5c02f0fb2fc81b584a4e171310f not found: ID does not exist"
Jul 24 15:13:04.777765 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.777742    2584 scope.go:117] "RemoveContainer" containerID="11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"
Jul 24 15:13:04.777986 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:13:04.777965    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20\": container with ID starting with 11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20 not found: ID does not exist" containerID="11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"
Jul 24 15:13:04.778077 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.777990    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20"} err="failed to get container status \"11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20\": rpc error: code = NotFound desc = could not find container \"11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20\": container with ID starting with 11b7f305260a644043eb8be3d97bdd92ba0acdbac54b3f59e3dba9ab75267b20 not found: ID does not exist"
Jul 24 15:13:04.781190 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.781169    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"]
Jul 24 15:13:04.785707 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:04.785684    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-891ff3-openshift-default-6fdcfff68b-ph7kf"]
Jul 24 15:13:05.670861 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:05.670814    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" path="/var/lib/kubelet/pods/ed59631e-d4f0-4c7e-ae80-b162164d9aa2/volumes"
Jul 24 15:13:05.899430 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:05.899408    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:06.071957 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:06.071921    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e27d422-d780-40e2-99bf-f75797fd8924-maas-api-service-ca\") pod \"4e27d422-d780-40e2-99bf-f75797fd8924\" (UID: \"4e27d422-d780-40e2-99bf-f75797fd8924\") "
Jul 24 15:13:06.072363 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:06.072339    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e27d422-d780-40e2-99bf-f75797fd8924-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e27d422-d780-40e2-99bf-f75797fd8924" (UID: "4e27d422-d780-40e2-99bf-f75797fd8924"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:13:06.173565 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:06.173537    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e27d422-d780-40e2-99bf-f75797fd8924-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:13:06.766879 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:06.766851    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"
Jul 24 15:13:06.767271 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:06.766888    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v" event={"ID":"4e27d422-d780-40e2-99bf-f75797fd8924","Type":"ContainerDied","Data":"8bf2ba0d0baa707e3e57f64dd7357d51acdd6fa09f36e6597e6d642c88b50afb"}
Jul 24 15:13:06.767271 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:06.766937    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bf2ba0d0baa707e3e57f64dd7357d51acdd6fa09f36e6597e6d642c88b50afb"
Jul 24 15:13:17.964495 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:17.964420    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:13:23.225456 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:23.225421    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"]
Jul 24 15:13:23.228680 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:23.228653    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-600d4a-bhm7v"]
Jul 24 15:13:23.671046 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:23.671018    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e27d422-d780-40e2-99bf-f75797fd8924" path="/var/lib/kubelet/pods/4e27d422-d780-40e2-99bf-f75797fd8924/volumes"
Jul 24 15:13:28.240044 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:13:28.240011    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:14:13.394518 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.394483    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"]
Jul 24 15:14:13.395034 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.394735    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" containerID="cri-o://64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee" gracePeriod=30
Jul 24 15:14:13.838056 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838021    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"]
Jul 24 15:14:13.838450 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838437    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e27d422-d780-40e2-99bf-f75797fd8924" containerName="revoke-keys"
Jul 24 15:14:13.838494 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838452    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e27d422-d780-40e2-99bf-f75797fd8924" containerName="revoke-keys"
Jul 24 15:14:13.838494 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838485    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy"
Jul 24 15:14:13.838494 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838491    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy"
Jul 24 15:14:13.838589 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838496    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy"
Jul 24 15:14:13.838589 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838503    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy"
Jul 24 15:14:13.838589 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838562    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy"
Jul 24 15:14:13.838589 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.838570    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e27d422-d780-40e2-99bf-f75797fd8924" containerName="revoke-keys"
Jul 24 15:14:13.841814 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.841790    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:13.844698 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.844675    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:14:13.844914 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.844901    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:14:13.849585 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.849563    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"]
Jul 24 15:14:13.903813 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:13.903790    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe245eb2-efa2-4cee-94fe-76fdbda7da30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4\" (UID: \"fe245eb2-efa2-4cee-94fe-76fdbda7da30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:14.005279 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:14.005246    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe245eb2-efa2-4cee-94fe-76fdbda7da30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4\" (UID: \"fe245eb2-efa2-4cee-94fe-76fdbda7da30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:14.005871 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:14.005853    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe245eb2-efa2-4cee-94fe-76fdbda7da30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4\" (UID: \"fe245eb2-efa2-4cee-94fe-76fdbda7da30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:14.153470 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:14.153393    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:14.290049 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:14.290024    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"]
Jul 24 15:14:14.293500 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:14:14.293457    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe245eb2_efa2_4cee_94fe_76fdbda7da30.slice/crio-5a4e7012bf66afd9a77268eaee538da860a8442c771e96f17eca88d172554680 WatchSource:0}: Error finding container 5a4e7012bf66afd9a77268eaee538da860a8442c771e96f17eca88d172554680: Status 404 returned error can't find the container with id 5a4e7012bf66afd9a77268eaee538da860a8442c771e96f17eca88d172554680
Jul 24 15:14:15.141867 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:15.141818    2584 generic.go:356] "Generic (PLEG): container finished" podID="fe245eb2-efa2-4cee-94fe-76fdbda7da30" containerID="d7e661ca9a3f454c0e0f3476a88397a4b84643dc59d9e49d293d668184d6572c" exitCode=0
Jul 24 15:14:15.142203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:15.141906    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4" event={"ID":"fe245eb2-efa2-4cee-94fe-76fdbda7da30","Type":"ContainerDied","Data":"d7e661ca9a3f454c0e0f3476a88397a4b84643dc59d9e49d293d668184d6572c"}
Jul 24 15:14:15.142203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:15.141957    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4" event={"ID":"fe245eb2-efa2-4cee-94fe-76fdbda7da30","Type":"ContainerStarted","Data":"5a4e7012bf66afd9a77268eaee538da860a8442c771e96f17eca88d172554680"}
Jul 24 15:14:16.282598 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:16.282574    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:16.324237 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:16.324213    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe245eb2-efa2-4cee-94fe-76fdbda7da30-maas-api-service-ca\") pod \"fe245eb2-efa2-4cee-94fe-76fdbda7da30\" (UID: \"fe245eb2-efa2-4cee-94fe-76fdbda7da30\") "
Jul 24 15:14:16.324624 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:16.324598    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe245eb2-efa2-4cee-94fe-76fdbda7da30-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe245eb2-efa2-4cee-94fe-76fdbda7da30" (UID: "fe245eb2-efa2-4cee-94fe-76fdbda7da30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:14:16.425785 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:16.425714    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe245eb2-efa2-4cee-94fe-76fdbda7da30-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:17.153043 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:17.153018    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"
Jul 24 15:14:17.153221 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:17.153055    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4" event={"ID":"fe245eb2-efa2-4cee-94fe-76fdbda7da30","Type":"ContainerDied","Data":"5a4e7012bf66afd9a77268eaee538da860a8442c771e96f17eca88d172554680"}
Jul 24 15:14:17.153221 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:17.153100    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a4e7012bf66afd9a77268eaee538da860a8442c771e96f17eca88d172554680"
Jul 24 15:14:17.521641 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:17.521605    2584 patch_prober.go:28] interesting pod/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 15:14:17.522008 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:17.521662    2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:14:18.654703 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.654645    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:14:18.745971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.745947    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istiod-ca-cert\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746117 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.745982    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrgww\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-kube-api-access-xrgww\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746117 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746027    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-envoy\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746117 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746067    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-token\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746117 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746088    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-credential-socket\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746117 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746114    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-socket\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746379 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746167    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-podinfo\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746379 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746203    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-certs\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746379 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746229    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-data\") pod \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\" (UID: \"57c7e964-a5d9-4d47-aa27-d9ef454723a6\") "
Jul 24 15:14:18.746379 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746304    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:14:18.746582 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746508    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:18.746682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746568    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:18.746739 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746702    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.746739 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746718    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.746739 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746731    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.746903 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.746863    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:18.747103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.747071    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:18.748344 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.748304    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:18.748448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.748401    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-kube-api-access-xrgww" (OuterVolumeSpecName: "kube-api-access-xrgww") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "kube-api-access-xrgww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:18.748673 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.748641    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:18.748673 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.748653    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "57c7e964-a5d9-4d47-aa27-d9ef454723a6" (UID: "57c7e964-a5d9-4d47-aa27-d9ef454723a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:14:18.847420 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.847388    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.847420 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.847414    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.847420 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.847423    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrgww\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-kube-api-access-xrgww\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.847420 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.847431    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.847680 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.847439    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/57c7e964-a5d9-4d47-aa27-d9ef454723a6-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:18.847680 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:18.847447    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/57c7e964-a5d9-4d47-aa27-d9ef454723a6-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:19.164929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.164891    2584 generic.go:356] "Generic (PLEG): container finished" podID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerID="64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee" exitCode=0
Jul 24 15:14:19.165126 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.164946    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" event={"ID":"57c7e964-a5d9-4d47-aa27-d9ef454723a6","Type":"ContainerDied","Data":"64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee"}
Jul 24 15:14:19.165126 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.164952    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"
Jul 24 15:14:19.165126 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.164987    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k" event={"ID":"57c7e964-a5d9-4d47-aa27-d9ef454723a6","Type":"ContainerDied","Data":"0c806d7dc54dc028ba44ab885aaa0b9e6433bad355ea120aecff01e126bd1784"}
Jul 24 15:14:19.165126 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.165008    2584 scope.go:117] "RemoveContainer" containerID="64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee"
Jul 24 15:14:19.174491 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.174474    2584 scope.go:117] "RemoveContainer" containerID="d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"
Jul 24 15:14:19.184970 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.184954    2584 scope.go:117] "RemoveContainer" containerID="64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee"
Jul 24 15:14:19.185213 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:14:19.185197    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee\": container with ID starting with 64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee not found: ID does not exist" containerID="64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee"
Jul 24 15:14:19.185270 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.185219    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee"} err="failed to get container status \"64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee\": rpc error: code = NotFound desc = could not find container \"64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee\": container with ID starting with 64c405d374517f213863551ec114b4d5ae9c1d7dc096249e40e147100fcf01ee not found: ID does not exist"
Jul 24 15:14:19.185270 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.185235    2584 scope.go:117] "RemoveContainer" containerID="d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"
Jul 24 15:14:19.185434 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:14:19.185420    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369\": container with ID starting with d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369 not found: ID does not exist" containerID="d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"
Jul 24 15:14:19.185472 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.185438    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369"} err="failed to get container status \"d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369\": rpc error: code = NotFound desc = could not find container \"d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369\": container with ID starting with d67af0191ae91c1a2c5dcc805a8a7f79e4403303c0fb80569ec8f1316425e369 not found: ID does not exist"
Jul 24 15:14:19.188730 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.188710    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"]
Jul 24 15:14:19.193346 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.193326    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-600d4a-openshift-default-69cbfb94bd-thk5k"]
Jul 24 15:14:19.670385 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:19.670352    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" path="/var/lib/kubelet/pods/57c7e964-a5d9-4d47-aa27-d9ef454723a6/volumes"
Jul 24 15:14:27.657529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:27.657494    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:14:28.937694 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:28.937660    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"]
Jul 24 15:14:28.940872 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:28.940849    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5c17a6-bh8g4"]
Jul 24 15:14:29.676946 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:29.676908    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe245eb2-efa2-4cee-94fe-76fdbda7da30" path="/var/lib/kubelet/pods/fe245eb2-efa2-4cee-94fe-76fdbda7da30/volumes"
Jul 24 15:14:36.506866 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:36.505996    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"]
Jul 24 15:14:36.506866 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:36.506348    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy" containerID="cri-o://ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207" gracePeriod=30
Jul 24 15:14:41.766152 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.766087    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:14:41.860721 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860686    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-credential-socket\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.860721 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860726    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-data\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860755    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r226h\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-kube-api-access-r226h\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860776    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5080429-445a-4b09-be98-693ff0cabe1f-istiod-ca-cert\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860805    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-certs\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860861    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5080429-445a-4b09-be98-693ff0cabe1f-istio-podinfo\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860890    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-socket\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860947    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-istio-token\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861007 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.860992    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-envoy\") pod \"a5080429-445a-4b09-be98-693ff0cabe1f\" (UID: \"a5080429-445a-4b09-be98-693ff0cabe1f\") "
Jul 24 15:14:41.861357 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861060    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:41.861357 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861244    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:41.861357 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861321    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:41.861529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861359    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5080429-445a-4b09-be98-693ff0cabe1f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:14:41.861529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861458    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.861529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861483    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.861529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861496    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.861669 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.861632    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:41.863098 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.863069    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-kube-api-access-r226h" (OuterVolumeSpecName: "kube-api-access-r226h") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "kube-api-access-r226h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:41.863216 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.863154    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:14:41.863263 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.863210    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5080429-445a-4b09-be98-693ff0cabe1f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:14:41.863360 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.863340    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5080429-445a-4b09-be98-693ff0cabe1f" (UID: "a5080429-445a-4b09-be98-693ff0cabe1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:14:41.962714 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.962683    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.962714 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.962707    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5080429-445a-4b09-be98-693ff0cabe1f-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.962714 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.962717    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r226h\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-kube-api-access-r226h\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.962971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.962726    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5080429-445a-4b09-be98-693ff0cabe1f-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.962971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.962736    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5080429-445a-4b09-be98-693ff0cabe1f-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:41.962971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:41.962744    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5080429-445a-4b09-be98-693ff0cabe1f-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:14:42.294736 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.294700    2584 generic.go:356] "Generic (PLEG): container finished" podID="a5080429-445a-4b09-be98-693ff0cabe1f" containerID="ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207" exitCode=0
Jul 24 15:14:42.294905 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.294758    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"
Jul 24 15:14:42.294905 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.294785    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" event={"ID":"a5080429-445a-4b09-be98-693ff0cabe1f","Type":"ContainerDied","Data":"ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207"}
Jul 24 15:14:42.294905 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.294883    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm" event={"ID":"a5080429-445a-4b09-be98-693ff0cabe1f","Type":"ContainerDied","Data":"9f2a4a84b729d6b9f86b0083b88eb3f536683f5edf9ae6e4b03896843e062daa"}
Jul 24 15:14:42.295044 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.294907    2584 scope.go:117] "RemoveContainer" containerID="ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207"
Jul 24 15:14:42.304252 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.304234    2584 scope.go:117] "RemoveContainer" containerID="0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"
Jul 24 15:14:42.312072 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.312054    2584 scope.go:117] "RemoveContainer" containerID="ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207"
Jul 24 15:14:42.312298 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:14:42.312283    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207\": container with ID starting with ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207 not found: ID does not exist" containerID="ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207"
Jul 24 15:14:42.312352 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.312305    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207"} err="failed to get container status \"ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207\": rpc error: code = NotFound desc = could not find container \"ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207\": container with ID starting with ff265a5f37f4b893fe83a7e1cb276b0190e23d643644c4bb476a0a9f1d97c207 not found: ID does not exist"
Jul 24 15:14:42.312352 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.312321    2584 scope.go:117] "RemoveContainer" containerID="0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"
Jul 24 15:14:42.312544 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:14:42.312528    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755\": container with ID starting with 0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755 not found: ID does not exist" containerID="0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"
Jul 24 15:14:42.312586 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.312548    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755"} err="failed to get container status \"0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755\": rpc error: code = NotFound desc = could not find container \"0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755\": container with ID starting with 0f3dac41d21b71d5692e075553f68d5afd3028b09cca4a011316297da688c755 not found: ID does not exist"
Jul 24 15:14:42.319036 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.319017    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"]
Jul 24 15:14:42.325856 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:42.325823    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5c17a6-openshift-default-64c6d58dc8-548dm"]
Jul 24 15:14:43.672598 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:43.672567    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" path="/var/lib/kubelet/pods/a5080429-445a-4b09-be98-693ff0cabe1f/volumes"
Jul 24 15:14:51.048450 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:14:51.048413    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:15:00.152580 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.152545    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"]
Jul 24 15:15:00.153186 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153162    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy"
Jul 24 15:15:00.153186 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153187    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153200    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153206    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153213    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe245eb2-efa2-4cee-94fe-76fdbda7da30" containerName="revoke-keys"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153219    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe245eb2-efa2-4cee-94fe-76fdbda7da30" containerName="revoke-keys"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153313    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153322    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed59631e-d4f0-4c7e-ae80-b162164d9aa2" containerName="istio-proxy"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153334    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy"
Jul 24 15:15:00.153339 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.153342    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe245eb2-efa2-4cee-94fe-76fdbda7da30" containerName="revoke-keys"
Jul 24 15:15:00.157847 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.157815    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"]
Jul 24 15:15:00.157959 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.157941    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"
Jul 24 15:15:00.158171 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.158157    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy"
Jul 24 15:15:00.158208 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.158175    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy"
Jul 24 15:15:00.158208 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.158189    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy"
Jul 24 15:15:00.158208 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.158198    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy"
Jul 24 15:15:00.158332 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.158315    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5080429-445a-4b09-be98-693ff0cabe1f" containerName="istio-proxy"
Jul 24 15:15:00.158332 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.158331    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="57c7e964-a5d9-4d47-aa27-d9ef454723a6" containerName="istio-proxy"
Jul 24 15:15:00.161068 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.160872    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n4c78\""
Jul 24 15:15:00.161250 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.161230    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"
Jul 24 15:15:00.163600 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.163575    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"]
Jul 24 15:15:00.168506 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.168485    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"]
Jul 24 15:15:00.173584 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.173567    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"
Jul 24 15:15:00.181676 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.181659    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"
Jul 24 15:15:00.346087 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.346061    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"]
Jul 24 15:15:00.347977 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:15:00.347949    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726e538e_f724_4238_a697_ac825c0ca7d8.slice/crio-3d0f4014e02bd1817771893ef77bb609dbc8fa495443f640b41a514bd1b6cf22 WatchSource:0}: Error finding container 3d0f4014e02bd1817771893ef77bb609dbc8fa495443f640b41a514bd1b6cf22: Status 404 returned error can't find the container with id 3d0f4014e02bd1817771893ef77bb609dbc8fa495443f640b41a514bd1b6cf22
Jul 24 15:15:00.405495 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.405434    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl" event={"ID":"726e538e-f724-4238-a697-ac825c0ca7d8","Type":"ContainerStarted","Data":"3d0f4014e02bd1817771893ef77bb609dbc8fa495443f640b41a514bd1b6cf22"}
Jul 24 15:15:00.530027 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.530000    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"]
Jul 24 15:15:00.531625 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:15:00.531592    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed35833_6d96_49e8_bc50_f51af9626658.slice/crio-21f50925eb64fb10d930a80cf60d01c22c32e9ec3625587fbb55eb1df16646d9 WatchSource:0}: Error finding container 21f50925eb64fb10d930a80cf60d01c22c32e9ec3625587fbb55eb1df16646d9: Status 404 returned error can't find the container with id 21f50925eb64fb10d930a80cf60d01c22c32e9ec3625587fbb55eb1df16646d9
Jul 24 15:15:00.836100 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.836067    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"]
Jul 24 15:15:00.836313 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:00.836286    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy" containerID="cri-o://72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2" gracePeriod=30
Jul 24 15:15:01.220790 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.220760    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"]
Jul 24 15:15:01.224378 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.224359    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:01.226718 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.226697    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:15:01.242008 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.241981    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"]
Jul 24 15:15:01.350475 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.350440    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e865d3e2-7127-424a-80b1-14954bf58199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx\" (UID: \"e865d3e2-7127-424a-80b1-14954bf58199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:01.411915 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.411872    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6" event={"ID":"6ed35833-6d96-49e8-bc50-f51af9626658","Type":"ContainerStarted","Data":"58676571a009cda62a6481b060cbfd6319fb9453c81af0aa8fdb3bfb7df1b736"}
Jul 24 15:15:01.412087 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.411935    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6" event={"ID":"6ed35833-6d96-49e8-bc50-f51af9626658","Type":"ContainerStarted","Data":"21f50925eb64fb10d930a80cf60d01c22c32e9ec3625587fbb55eb1df16646d9"}
Jul 24 15:15:01.413425 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.413401    2584 generic.go:356] "Generic (PLEG): container finished" podID="726e538e-f724-4238-a697-ac825c0ca7d8" containerID="8b5ce56ed4e4d4a1e254670251f259eda753c793d76d3ed5ad7cd5d97c20745b" exitCode=0
Jul 24 15:15:01.413530 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.413461    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl" event={"ID":"726e538e-f724-4238-a697-ac825c0ca7d8","Type":"ContainerDied","Data":"8b5ce56ed4e4d4a1e254670251f259eda753c793d76d3ed5ad7cd5d97c20745b"}
Jul 24 15:15:01.428529 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.428485    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6" podStartSLOduration=1.428469385 podStartE2EDuration="1.428469385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:01.427903218 +0000 UTC m=+3158.212603682" watchObservedRunningTime="2026-07-24 15:15:01.428469385 +0000 UTC m=+3158.213169850"
Jul 24 15:15:01.451057 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.451030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e865d3e2-7127-424a-80b1-14954bf58199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx\" (UID: \"e865d3e2-7127-424a-80b1-14954bf58199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:01.451608 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.451588    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e865d3e2-7127-424a-80b1-14954bf58199-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx\" (UID: \"e865d3e2-7127-424a-80b1-14954bf58199\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:01.543712 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.543679    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:01.697932 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:01.697910    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"]
Jul 24 15:15:01.699698 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:15:01.699674    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode865d3e2_7127_424a_80b1_14954bf58199.slice/crio-c8b9934beb0547eb8e1defc4ba461af691d81d726baf4617cb88a16c16bbfd10 WatchSource:0}: Error finding container c8b9934beb0547eb8e1defc4ba461af691d81d726baf4617cb88a16c16bbfd10: Status 404 returned error can't find the container with id c8b9934beb0547eb8e1defc4ba461af691d81d726baf4617cb88a16c16bbfd10
Jul 24 15:15:02.420516 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:02.420490    2584 generic.go:356] "Generic (PLEG): container finished" podID="e865d3e2-7127-424a-80b1-14954bf58199" containerID="d1be4498a32e0d3b5a5e53c48beb02764f4ebaf7dd301ad7831b38f6b75c63f7" exitCode=0
Jul 24 15:15:02.420889 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:02.420577    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx" event={"ID":"e865d3e2-7127-424a-80b1-14954bf58199","Type":"ContainerDied","Data":"d1be4498a32e0d3b5a5e53c48beb02764f4ebaf7dd301ad7831b38f6b75c63f7"}
Jul 24 15:15:02.420889 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:02.420631    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx" event={"ID":"e865d3e2-7127-424a-80b1-14954bf58199","Type":"ContainerStarted","Data":"c8b9934beb0547eb8e1defc4ba461af691d81d726baf4617cb88a16c16bbfd10"}
Jul 24 15:15:02.422117 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:02.422095    2584 generic.go:356] "Generic (PLEG): container finished" podID="6ed35833-6d96-49e8-bc50-f51af9626658" containerID="58676571a009cda62a6481b060cbfd6319fb9453c81af0aa8fdb3bfb7df1b736" exitCode=0
Jul 24 15:15:02.422216 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:02.422144    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6" event={"ID":"6ed35833-6d96-49e8-bc50-f51af9626658","Type":"ContainerDied","Data":"58676571a009cda62a6481b060cbfd6319fb9453c81af0aa8fdb3bfb7df1b736"}
Jul 24 15:15:02.562516 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:02.562490    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"
Jul 24 15:15:03.429100 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.429060    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl" event={"ID":"726e538e-f724-4238-a697-ac825c0ca7d8","Type":"ContainerDied","Data":"3d0f4014e02bd1817771893ef77bb609dbc8fa495443f640b41a514bd1b6cf22"}
Jul 24 15:15:03.429512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.429122    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d0f4014e02bd1817771893ef77bb609dbc8fa495443f640b41a514bd1b6cf22"
Jul 24 15:15:03.429512 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.429076    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"
Jul 24 15:15:03.561785 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.561757    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"
Jul 24 15:15:03.602609 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.602589    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:03.671441 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.671413    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e865d3e2-7127-424a-80b1-14954bf58199-maas-api-service-ca\") pod \"e865d3e2-7127-424a-80b1-14954bf58199\" (UID: \"e865d3e2-7127-424a-80b1-14954bf58199\") "
Jul 24 15:15:03.671788 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.671769    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e865d3e2-7127-424a-80b1-14954bf58199-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e865d3e2-7127-424a-80b1-14954bf58199" (UID: "e865d3e2-7127-424a-80b1-14954bf58199"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:15:03.772588 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:03.772565    2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e865d3e2-7127-424a-80b1-14954bf58199-maas-api-service-ca\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:04.435926 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:04.435896    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"
Jul 24 15:15:04.436348 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:04.435941    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx" event={"ID":"e865d3e2-7127-424a-80b1-14954bf58199","Type":"ContainerDied","Data":"c8b9934beb0547eb8e1defc4ba461af691d81d726baf4617cb88a16c16bbfd10"}
Jul 24 15:15:04.436348 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:04.435995    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8b9934beb0547eb8e1defc4ba461af691d81d726baf4617cb88a16c16bbfd10"
Jul 24 15:15:04.437264 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:04.437248    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6"
Jul 24 15:15:04.437358 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:04.437245    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748435-k8vz6" event={"ID":"6ed35833-6d96-49e8-bc50-f51af9626658","Type":"ContainerDied","Data":"21f50925eb64fb10d930a80cf60d01c22c32e9ec3625587fbb55eb1df16646d9"}
Jul 24 15:15:04.437404 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:04.437391    2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21f50925eb64fb10d930a80cf60d01c22c32e9ec3625587fbb55eb1df16646d9"
Jul 24 15:15:06.107535 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.107479    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:15:06.197741 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197709    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-socket\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.197971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197754    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-credential-socket\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.197971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197782    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxrbf\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-kube-api-access-zxrbf\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.197971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197868    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-istio-token\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.197971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197909    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-data\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.197971 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197956    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-certs\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.198257 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.197992    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-envoy\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.198257 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198026    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1376081d-75c9-499c-b274-417f6fab91a0-istio-podinfo\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.198257 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198068    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:06.198257 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198097    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:06.198257 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198089    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1376081d-75c9-499c-b274-417f6fab91a0-istiod-ca-cert\") pod \"1376081d-75c9-499c-b274-417f6fab91a0\" (UID: \"1376081d-75c9-499c-b274-417f6fab91a0\") "
Jul 24 15:15:06.198257 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198219    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:06.198533 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198417    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1376081d-75c9-499c-b274-417f6fab91a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:15:06.198533 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198441    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.198533 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198459    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.198533 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198476    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.198941 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.198917    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:06.200659 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.200629    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-kube-api-access-zxrbf" (OuterVolumeSpecName: "kube-api-access-zxrbf") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "kube-api-access-zxrbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:06.200746 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.200661    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:06.200746 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.200661    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1376081d-75c9-499c-b274-417f6fab91a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:15:06.200746 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.200674    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1376081d-75c9-499c-b274-417f6fab91a0" (UID: "1376081d-75c9-499c-b274-417f6fab91a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:06.298967 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.298931    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1376081d-75c9-499c-b274-417f6fab91a0-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.298967 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.298959    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1376081d-75c9-499c-b274-417f6fab91a0-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.298967 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.298968    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxrbf\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-kube-api-access-zxrbf\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.298967 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.298977    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1376081d-75c9-499c-b274-417f6fab91a0-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.299234 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.298986    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.299234 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.298994    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1376081d-75c9-499c-b274-417f6fab91a0-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:06.450138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.450043    2584 generic.go:356] "Generic (PLEG): container finished" podID="1376081d-75c9-499c-b274-417f6fab91a0" containerID="72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2" exitCode=0
Jul 24 15:15:06.450138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.450094    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"
Jul 24 15:15:06.450138 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.450122    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" event={"ID":"1376081d-75c9-499c-b274-417f6fab91a0","Type":"ContainerDied","Data":"72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2"}
Jul 24 15:15:06.450354 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.450181    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf" event={"ID":"1376081d-75c9-499c-b274-417f6fab91a0","Type":"ContainerDied","Data":"a807a28e403077f1502c8eff05564674ce2b57ad48edd2fc78b41196f13f1321"}
Jul 24 15:15:06.450354 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.450212    2584 scope.go:117] "RemoveContainer" containerID="72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2"
Jul 24 15:15:06.459723 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.459685    2584 scope.go:117] "RemoveContainer" containerID="f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"
Jul 24 15:15:06.468347 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.468327    2584 scope.go:117] "RemoveContainer" containerID="72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2"
Jul 24 15:15:06.468619 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:15:06.468596    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2\": container with ID starting with 72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2 not found: ID does not exist" containerID="72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2"
Jul 24 15:15:06.468704 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.468623    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2"} err="failed to get container status \"72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2\": rpc error: code = NotFound desc = could not find container \"72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2\": container with ID starting with 72fc19f38e431151e3f9dd598e918cbc104297629018a4e013d8be08dbf3feb2 not found: ID does not exist"
Jul 24 15:15:06.468704 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.468640    2584 scope.go:117] "RemoveContainer" containerID="f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"
Jul 24 15:15:06.468912 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:15:06.468893    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a\": container with ID starting with f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a not found: ID does not exist" containerID="f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"
Jul 24 15:15:06.469000 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.468918    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a"} err="failed to get container status \"f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a\": rpc error: code = NotFound desc = could not find container \"f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a\": container with ID starting with f90a402068eb07026d3908354ec7ef7de16b2c7fa050951fc57712826dc1158a not found: ID does not exist"
Jul 24 15:15:06.475828 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.475808    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"]
Jul 24 15:15:06.484806 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:06.484784    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-876f50-openshift-default-8668566fbd-5g8pf"]
Jul 24 15:15:07.670988 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:07.670953    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1376081d-75c9-499c-b274-417f6fab91a0" path="/var/lib/kubelet/pods/1376081d-75c9-499c-b274-417f6fab91a0/volumes"
Jul 24 15:15:11.989797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:11.989765    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"]
Jul 24 15:15:11.992742 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:11.992721    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-0d170b-29748435-2j5gl"]
Jul 24 15:15:13.672019 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:13.671982    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="726e538e-f724-4238-a697-ac825c0ca7d8" path="/var/lib/kubelet/pods/726e538e-f724-4238-a697-ac825c0ca7d8/volumes"
Jul 24 15:15:14.888369 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:14.888335    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qhrnr"]
Jul 24 15:15:16.313477 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:16.313445    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"]
Jul 24 15:15:16.317818 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:16.317797    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0d170b-dcwnx"]
Jul 24 15:15:17.677037 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:17.677004    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e865d3e2-7127-424a-80b1-14954bf58199" path="/var/lib/kubelet/pods/e865d3e2-7127-424a-80b1-14954bf58199/volumes"
Jul 24 15:15:23.857873 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:23.857823    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"]
Jul 24 15:15:23.858424 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:23.858119    2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy" containerID="cri-o://018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45" gracePeriod=30
Jul 24 15:15:29.117383 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.117353    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:15:29.189879 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.189798    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-data\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.189879 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.189827    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-envoy\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.189893    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-token\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.189923    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-credential-socket\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.189965    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istiod-ca-cert\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.189994    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-socket\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190022    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-certs\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190059    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgghz\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-kube-api-access-hgghz\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190376 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190087    2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-podinfo\") pod \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\" (UID: \"16b5ef37-6a59-4852-b3b2-0669c1f6f354\") "
Jul 24 15:15:29.190376 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190216    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:29.190487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190370    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:29.190487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190390    2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-credential-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.190487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190426    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:15:29.190604 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190484    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:29.190794 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.190774    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-data" (OuterVolumeSpecName: "istio-data") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:29.192176 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.192150    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-token" (OuterVolumeSpecName: "istio-token") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:29.192288 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.192257    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-kube-api-access-hgghz" (OuterVolumeSpecName: "kube-api-access-hgghz") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "kube-api-access-hgghz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:15:29.192465 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.192447    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:15:29.192570 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.192554    2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "16b5ef37-6a59-4852-b3b2-0669c1f6f354" (UID: "16b5ef37-6a59-4852-b3b2-0669c1f6f354"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 15:15:29.291565 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291537    2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-token\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291565 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291564    2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istiod-ca-cert\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291565 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291575    2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-socket\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291583    2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-workload-certs\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291592    2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgghz\" (UniqueName: \"kubernetes.io/projected/16b5ef37-6a59-4852-b3b2-0669c1f6f354-kube-api-access-hgghz\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291602    2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-podinfo\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291609    2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-data\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.291797 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.291617    2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16b5ef37-6a59-4852-b3b2-0669c1f6f354-istio-envoy\") on node \"ip-10-0-135-93.ec2.internal\" DevicePath \"\""
Jul 24 15:15:29.576427 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.576391    2584 generic.go:356] "Generic (PLEG): container finished" podID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerID="018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45" exitCode=0
Jul 24 15:15:29.576587 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.576426    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" event={"ID":"16b5ef37-6a59-4852-b3b2-0669c1f6f354","Type":"ContainerDied","Data":"018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45"}
Jul 24 15:15:29.576587 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.576483    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp" event={"ID":"16b5ef37-6a59-4852-b3b2-0669c1f6f354","Type":"ContainerDied","Data":"5282ef744df0a7d5c01647571a4f5f95f5b6e0aa37de960c22c7835486ef6ac2"}
Jul 24 15:15:29.576587 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.576439    2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"
Jul 24 15:15:29.576587 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.576509    2584 scope.go:117] "RemoveContainer" containerID="018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45"
Jul 24 15:15:29.592016 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.591998    2584 scope.go:117] "RemoveContainer" containerID="65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"
Jul 24 15:15:29.600217 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.600199    2584 scope.go:117] "RemoveContainer" containerID="018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45"
Jul 24 15:15:29.600443 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:15:29.600424    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45\": container with ID starting with 018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45 not found: ID does not exist" containerID="018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45"
Jul 24 15:15:29.600532 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.600454    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45"} err="failed to get container status \"018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45\": rpc error: code = NotFound desc = could not find container \"018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45\": container with ID starting with 018e95a903b14378bfbc56c25bf5aa587e2cad452ce6e3800cb951bc6d0e0f45 not found: ID does not exist"
Jul 24 15:15:29.600532 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.600478    2584 scope.go:117] "RemoveContainer" containerID="65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"
Jul 24 15:15:29.600738 ip-10-0-135-93 kubenswrapper[2584]: E0724 15:15:29.600720    2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6\": container with ID starting with 65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6 not found: ID does not exist" containerID="65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"
Jul 24 15:15:29.600778 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.600745    2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6"} err="failed to get container status \"65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6\": rpc error: code = NotFound desc = could not find container \"65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6\": container with ID starting with 65eab7160212e3cda00d002a1f762d0802000e55e1a6e0ac9c5d66e7269b0dd6 not found: ID does not exist"
Jul 24 15:15:29.606848 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.606810    2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"]
Jul 24 15:15:29.614313 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.614292    2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0d170b-openshift-default-bd56d77d8-j9ncp"]
Jul 24 15:15:29.670777 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:29.670754    2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" path="/var/lib/kubelet/pods/16b5ef37-6a59-4852-b3b2-0669c1f6f354/volumes"
Jul 24 15:15:30.507633 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:30.507603    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-74vtf_6b265693-dfd3-4099-a377-4bfaf207ad57/manager/0.log"
Jul 24 15:15:30.638182 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:30.638150    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-j98hz_d6dbdcdc-fd02-4e07-b4a8-b05aca23a968/manager/0.log"
Jul 24 15:15:30.891842 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:30.891816    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4mrl2_6b667428-c4e6-4e34-8682-7f48060e6526/server/0.log"
Jul 24 15:15:31.027067 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:31.027035    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fvgbg_410dbd66-021d-43a4-b47b-471cee57ab7d/manager/0.log"
Jul 24 15:15:31.419802 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:31.419774    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bbcbb69d-v9hj7_6f858af5-17df-41c1-ad70-20ade164f31c/copy-manifests/0.log"
Jul 24 15:15:31.439882 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:31.439826    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bbcbb69d-v9hj7_6f858af5-17df-41c1-ad70-20ade164f31c/manager/0.log"
Jul 24 15:15:31.948545 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:31.948518    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748405-gxphp_714c95c2-0de5-4c6e-8619-4b653035235d/cleanup/0.log"
Jul 24 15:15:32.071661 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:32.071631    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748420-zttn8_9f2d85a6-49ec-458a-afb7-328c899e77d9/cleanup/0.log"
Jul 24 15:15:32.195259 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:32.195229    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748435-k8vz6_6ed35833-6d96-49e8-bc50-f51af9626658/cleanup/0.log"
Jul 24 15:15:33.990579 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:33.990549    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5jxjq_1b034609-fb4a-462a-ac66-825aa481033c/manager/0.log"
Jul 24 15:15:34.245127 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:34.245054    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hsxvv_f9e220d2-b7d2-45c4-8021-94b9dd7b34d1/registry-server/0.log"
Jul 24 15:15:34.377926 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:34.377902    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nz76j_246d5a54-7d04-4a20-ae5c-fa66c1e96d16/manager/0.log"
Jul 24 15:15:34.494541 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:34.494516    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qhrnr_3968abd3-b4b5-4f9e-8c74-b45738a10c5d/limitador/0.log"
Jul 24 15:15:34.631559 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:34.631536    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vp9j7_f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d/manager/0.log"
Jul 24 15:15:35.031154 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:35.031128    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq_642f2c92-f994-4277-8613-d7371432db23/istio-proxy/0.log"
Jul 24 15:15:35.401523 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:35.401439    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b7bd974d-7rzwd_5483efb5-b239-4f23-92ad-1d3253577f4e/kube-auth-proxy/0.log"
Jul 24 15:15:35.538682 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:35.538648    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/1.log"
Jul 24 15:15:35.931848 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:35.931812    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-df9867497-4w8nq_6eeb8c69-5d01-44e5-9fa3-77a9687c007b/router/0.log"
Jul 24 15:15:42.825049 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825004    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-44pkg/must-gather-99htb"]
Jul 24 15:15:42.825576 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825557    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed35833-6d96-49e8-bc50-f51af9626658" containerName="cleanup"
Jul 24 15:15:42.825647 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825580    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed35833-6d96-49e8-bc50-f51af9626658" containerName="cleanup"
Jul 24 15:15:42.825647 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825597    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy"
Jul 24 15:15:42.825647 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825606    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy"
Jul 24 15:15:42.825647 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825634    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="726e538e-f724-4238-a697-ac825c0ca7d8" containerName="cleanup"
Jul 24 15:15:42.825647 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825642    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="726e538e-f724-4238-a697-ac825c0ca7d8" containerName="cleanup"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825653    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825663    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825677    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e865d3e2-7127-424a-80b1-14954bf58199" containerName="revoke-keys"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825686    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e865d3e2-7127-424a-80b1-14954bf58199" containerName="revoke-keys"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825706    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825714    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825814    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825827    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e865d3e2-7127-424a-80b1-14954bf58199" containerName="revoke-keys"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825854    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825862    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="726e538e-f724-4238-a697-ac825c0ca7d8" containerName="cleanup"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825873    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825881    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed35833-6d96-49e8-bc50-f51af9626658" containerName="cleanup"
Jul 24 15:15:42.825929 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825890    2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1376081d-75c9-499c-b274-417f6fab91a0" containerName="istio-proxy"
Jul 24 15:15:42.826549 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.825992    2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy"
Jul 24 15:15:42.826549 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.826002    2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="16b5ef37-6a59-4852-b3b2-0669c1f6f354" containerName="istio-proxy"
Jul 24 15:15:42.829446 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.829425    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:42.832487 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.832468    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-44pkg\"/\"openshift-service-ca.crt\""
Jul 24 15:15:42.832570 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.832478    2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-44pkg\"/\"default-dockercfg-6psc7\""
Jul 24 15:15:42.833043 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.833025    2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-44pkg\"/\"kube-root-ca.crt\""
Jul 24 15:15:42.837285 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.837266    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44pkg/must-gather-99htb"]
Jul 24 15:15:42.914855 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.914811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1f3b7605-575d-4a7c-8250-d111d2a33393-must-gather-output\") pod \"must-gather-99htb\" (UID: \"1f3b7605-575d-4a7c-8250-d111d2a33393\") " pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:42.914976 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:42.914918    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbbvb\" (UniqueName: \"kubernetes.io/projected/1f3b7605-575d-4a7c-8250-d111d2a33393-kube-api-access-jbbvb\") pod \"must-gather-99htb\" (UID: \"1f3b7605-575d-4a7c-8250-d111d2a33393\") " pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:43.015741 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.015717    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1f3b7605-575d-4a7c-8250-d111d2a33393-must-gather-output\") pod \"must-gather-99htb\" (UID: \"1f3b7605-575d-4a7c-8250-d111d2a33393\") " pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:43.015863 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.015768    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbbvb\" (UniqueName: \"kubernetes.io/projected/1f3b7605-575d-4a7c-8250-d111d2a33393-kube-api-access-jbbvb\") pod \"must-gather-99htb\" (UID: \"1f3b7605-575d-4a7c-8250-d111d2a33393\") " pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:43.016113 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.016092    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1f3b7605-575d-4a7c-8250-d111d2a33393-must-gather-output\") pod \"must-gather-99htb\" (UID: \"1f3b7605-575d-4a7c-8250-d111d2a33393\") " pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:43.025320 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.025304    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbbvb\" (UniqueName: \"kubernetes.io/projected/1f3b7605-575d-4a7c-8250-d111d2a33393-kube-api-access-jbbvb\") pod \"must-gather-99htb\" (UID: \"1f3b7605-575d-4a7c-8250-d111d2a33393\") " pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:43.139515 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.139439    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44pkg/must-gather-99htb"
Jul 24 15:15:43.280281 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.280258    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44pkg/must-gather-99htb"]
Jul 24 15:15:43.281785 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:15:43.281757    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f3b7605_575d_4a7c_8250_d111d2a33393.slice/crio-e969d2a90ea064ce57af5570e24f461a68671913ac0564b7c4a1f6e600e1e4e6 WatchSource:0}: Error finding container e969d2a90ea064ce57af5570e24f461a68671913ac0564b7c4a1f6e600e1e4e6: Status 404 returned error can't find the container with id e969d2a90ea064ce57af5570e24f461a68671913ac0564b7c4a1f6e600e1e4e6
Jul 24 15:15:43.283600 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.283580    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:15:43.655619 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:43.655589    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44pkg/must-gather-99htb" event={"ID":"1f3b7605-575d-4a7c-8250-d111d2a33393","Type":"ContainerStarted","Data":"e969d2a90ea064ce57af5570e24f461a68671913ac0564b7c4a1f6e600e1e4e6"}
Jul 24 15:15:44.667939 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:44.667888    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44pkg/must-gather-99htb" event={"ID":"1f3b7605-575d-4a7c-8250-d111d2a33393","Type":"ContainerStarted","Data":"f3b8d7d040e895209e7b67fd184bdf207ce793b524e6f632777f2edbe334aa08"}
Jul 24 15:15:44.668514 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:44.668489    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44pkg/must-gather-99htb" event={"ID":"1f3b7605-575d-4a7c-8250-d111d2a33393","Type":"ContainerStarted","Data":"7f32d1d71d5a3801d80b0baae2d509d277b4f59ce68613f44ba91efc01339406"}
Jul 24 15:15:44.702319 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:44.702261    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-44pkg/must-gather-99htb" podStartSLOduration=2.042487552 podStartE2EDuration="2.702241747s" totalImagesPullingTime="659.754195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:44.700795926 +0000 UTC m=+3201.485496394" watchObservedRunningTime="2026-07-24 15:15:44.702241747 +0000 UTC m=+3201.486942212"
Jul 24 15:15:45.709049 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:45.709013    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wz7zb_8d36989a-3602-4e05-8ab1-7f85407fd61c/global-pull-secret-syncer/0.log"
Jul 24 15:15:45.868904 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:45.868864    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lmt58_366e9dfd-b78f-4684-8649-cb33cfdd0e4f/konnectivity-agent/0.log"
Jul 24 15:15:45.996596 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:45.996524    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-93.ec2.internal_feea928f3452f52f53864c7f7d2090cd/haproxy/0.log"
Jul 24 15:15:50.992698 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:50.992664    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5jxjq_1b034609-fb4a-462a-ac66-825aa481033c/manager/0.log"
Jul 24 15:15:51.118854 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:51.118493    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hsxvv_f9e220d2-b7d2-45c4-8021-94b9dd7b34d1/registry-server/0.log"
Jul 24 15:15:51.320595 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:51.320562    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nz76j_246d5a54-7d04-4a20-ae5c-fa66c1e96d16/manager/0.log"
Jul 24 15:15:51.347561 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:51.347533    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qhrnr_3968abd3-b4b5-4f9e-8c74-b45738a10c5d/limitador/0.log"
Jul 24 15:15:51.460065 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:51.460030    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vp9j7_f7f47e37-2d8f-4485-a9f5-2cdb3c56fa5d/manager/0.log"
Jul 24 15:15:53.388278 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:53.388240    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-kschn_6852cf8e-b13e-49e3-b9bb-383a5ea7e803/cluster-monitoring-operator/0.log"
Jul 24 15:15:53.702938 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:53.702851    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6j7x_0e97c5be-4c6b-4e1f-9eff-d517f9f63dee/node-exporter/0.log"
Jul 24 15:15:53.728350 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:53.728324    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6j7x_0e97c5be-4c6b-4e1f-9eff-d517f9f63dee/kube-rbac-proxy/0.log"
Jul 24 15:15:53.758498 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:53.758472    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6j7x_0e97c5be-4c6b-4e1f-9eff-d517f9f63dee/init-textfile/0.log"
Jul 24 15:15:55.662808 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:55.662779    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-k6dbg_d0522bad-1a97-410d-bbb2-0037cdc9852d/networking-console-plugin/0.log"
Jul 24 15:15:55.904432 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:55.904398    2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"]
Jul 24 15:15:55.913254 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:55.913164    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:55.915082 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:55.915047    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"]
Jul 24 15:15:56.049696 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.049648    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kdjz\" (UniqueName: \"kubernetes.io/projected/8009ad15-9457-459f-8331-0400a315e9c6-kube-api-access-9kdjz\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.050058 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.050035    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-podres\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.050228 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.050214    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-sys\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.050455 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.050441    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-lib-modules\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.050611 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.050596    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-proc\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151412 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151372    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-podres\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151593 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151425    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-sys\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151593 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151534    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-lib-modules\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151593 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-podres\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151593 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151583    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-proc\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151810 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kdjz\" (UniqueName: \"kubernetes.io/projected/8009ad15-9457-459f-8331-0400a315e9c6-kube-api-access-9kdjz\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151810 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151643    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-proc\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151810 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151560    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-sys\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.151810 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.151703    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8009ad15-9457-459f-8331-0400a315e9c6-lib-modules\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.161321 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.161295    2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kdjz\" (UniqueName: \"kubernetes.io/projected/8009ad15-9457-459f-8331-0400a315e9c6-kube-api-access-9kdjz\") pod \"perf-node-gather-daemonset-2ttww\" (UID: \"8009ad15-9457-459f-8331-0400a315e9c6\") " pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.226433 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.226352    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.304806 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.304722    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/1.log"
Jul 24 15:15:56.314679 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.314604    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fxg9p_74e9f2bd-c80c-49d5-a2ee-19e94fc8ddd7/console-operator/2.log"
Jul 24 15:15:56.407486 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.406325    2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"]
Jul 24 15:15:56.411322 ip-10-0-135-93 kubenswrapper[2584]: W0724 15:15:56.411274    2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8009ad15_9457_459f_8331_0400a315e9c6.slice/crio-eb6ceb76bb6250cf8c7078df686f1effbc2ec979cebd44c20b4822ba00c247de WatchSource:0}: Error finding container eb6ceb76bb6250cf8c7078df686f1effbc2ec979cebd44c20b4822ba00c247de: Status 404 returned error can't find the container with id eb6ceb76bb6250cf8c7078df686f1effbc2ec979cebd44c20b4822ba00c247de
Jul 24 15:15:56.761617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.761519    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww" event={"ID":"8009ad15-9457-459f-8331-0400a315e9c6","Type":"ContainerStarted","Data":"e33cd9f304452313fe61ebc35b0e77f80d5a2980fc929cf6da3ce103f6792a72"}
Jul 24 15:15:56.761617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.761593    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:15:56.761617 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.761615    2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww" event={"ID":"8009ad15-9457-459f-8331-0400a315e9c6","Type":"ContainerStarted","Data":"eb6ceb76bb6250cf8c7078df686f1effbc2ec979cebd44c20b4822ba00c247de"}
Jul 24 15:15:56.781960 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:56.781311    2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww" podStartSLOduration=1.7812922960000002 podStartE2EDuration="1.781292296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:15:56.778724627 +0000 UTC m=+3213.563425091" watchObservedRunningTime="2026-07-24 15:15:56.781292296 +0000 UTC m=+3213.565992761"
Jul 24 15:15:57.481584 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:57.481551    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-hcrwc_0668758d-4f81-4136-a78a-1f170490d401/volume-data-source-validator/0.log"
Jul 24 15:15:58.690937 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:58.690908    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ssbjq_c872d451-23b6-49e7-b8b0-a41cdcbb4ff5/dns/0.log"
Jul 24 15:15:58.715730 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:58.715697    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ssbjq_c872d451-23b6-49e7-b8b0-a41cdcbb4ff5/kube-rbac-proxy/0.log"
Jul 24 15:15:58.826705 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:58.826675    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2htf_be0981fc-a012-4c1d-8282-653973ba75b5/dns-node-resolver/0.log"
Jul 24 15:15:59.499433 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:15:59.499403    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hpn4m_736d187e-5b4a-41b9-a918-a723cec1dcd7/node-ca/0.log"
Jul 24 15:16:00.581785 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:00.581751    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq_642f2c92-f994-4277-8613-d7371432db23/istio-proxy/0.log"
Jul 24 15:16:00.781827 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:00.781798    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b7bd974d-7rzwd_5483efb5-b239-4f23-92ad-1d3253577f4e/kube-auth-proxy/0.log"
Jul 24 15:16:00.816361 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:00.816331    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/0.log"
Jul 24 15:16:00.871203 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:00.871129    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/1.log"
Jul 24 15:16:00.981127 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:00.981096    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-df9867497-4w8nq_6eeb8c69-5d01-44e5-9fa3-77a9687c007b/router/0.log"
Jul 24 15:16:01.807222 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:01.807158    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbkpq_ea5c9656-f807-497c-8f41-715f74b4385f/serve-healthcheck-canary/0.log"
Jul 24 15:16:02.385328 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:02.385297    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-j9lx2_a09f2c2a-eb19-440c-8cf2-24d6f4328003/insights-operator/0.log"
Jul 24 15:16:02.386550 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:02.386525    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-j9lx2_a09f2c2a-eb19-440c-8cf2-24d6f4328003/insights-operator/1.log"
Jul 24 15:16:02.598536 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:02.598509    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t6jpv_39a5116a-222d-4c6e-98b6-9ebc465096f4/kube-rbac-proxy/0.log"
Jul 24 15:16:02.621851 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:02.621803    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t6jpv_39a5116a-222d-4c6e-98b6-9ebc465096f4/exporter/0.log"
Jul 24 15:16:02.646666 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:02.646599    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t6jpv_39a5116a-222d-4c6e-98b6-9ebc465096f4/extractor/0.log"
Jul 24 15:16:02.781073 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:02.780392    2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-44pkg/perf-node-gather-daemonset-2ttww"
Jul 24 15:16:05.005766 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:05.005736    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-74vtf_6b265693-dfd3-4099-a377-4bfaf207ad57/manager/0.log"
Jul 24 15:16:05.432361 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:05.432132    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-j98hz_d6dbdcdc-fd02-4e07-b4a8-b05aca23a968/manager/0.log"
Jul 24 15:16:05.636141 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:05.636114    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4mrl2_6b667428-c4e6-4e34-8682-7f48060e6526/server/0.log"
Jul 24 15:16:05.779856 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:05.778914    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fvgbg_410dbd66-021d-43a4-b47b-471cee57ab7d/manager/0.log"
Jul 24 15:16:06.067936 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.067890    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bbcbb69d-v9hj7_6f858af5-17df-41c1-ad70-20ade164f31c/manager/0.log"
Jul 24 15:16:06.082905 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.082868    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq_642f2c92-f994-4277-8613-d7371432db23/istio-proxy/0.log"
Jul 24 15:16:06.089289 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.089232    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64bbcbb69d-v9hj7_6f858af5-17df-41c1-ad70-20ade164f31c/copy-manifests/0.log"
Jul 24 15:16:06.269103 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.269068    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b7bd974d-7rzwd_5483efb5-b239-4f23-92ad-1d3253577f4e/kube-auth-proxy/0.log"
Jul 24 15:16:06.302928 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.302899    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/0.log"
Jul 24 15:16:06.355001 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.354929    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/1.log"
Jul 24 15:16:06.419311 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:06.419285    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-df9867497-4w8nq_6eeb8c69-5d01-44e5-9fa3-77a9687c007b/router/0.log"
Jul 24 15:16:07.880448 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:07.880365    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-57jxq_642f2c92-f994-4277-8613-d7371432db23/istio-proxy/0.log"
Jul 24 15:16:08.053347 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:08.053318    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68b7bd974d-7rzwd_5483efb5-b239-4f23-92ad-1d3253577f4e/kube-auth-proxy/0.log"
Jul 24 15:16:08.079788 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:08.079759    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/0.log"
Jul 24 15:16:08.134328 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:08.134249    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-7dwxz_3c712d07-40d1-4f63-837f-26a2ee2dc27f/istio-proxy/1.log"
Jul 24 15:16:08.208709 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:08.208676    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-df9867497-4w8nq_6eeb8c69-5d01-44e5-9fa3-77a9687c007b/router/0.log"
Jul 24 15:16:12.631969 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:12.631938    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fgg7k_d66d09b5-1150-470e-818e-1eed339010cb/migrator/0.log"
Jul 24 15:16:12.658630 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:12.658592    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fgg7k_d66d09b5-1150-470e-818e-1eed339010cb/graceful-termination/0.log"
Jul 24 15:16:13.073696 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:13.073662    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c_5d4ee50f-92ce-43fe-92f2-69391d789624/kube-storage-version-migrator-operator/1.log"
Jul 24 15:16:13.075090 ip-10-0-135-93 kubenswrapper[2584]: I0724 15:16:13.075064    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-zww5c_5d4ee50f-92ce-43fe-92f2-69391d789624/kube-storage-version-migrator-operator/0.log"