Jul 27 19:26:19.409566 ip-10-0-128-237 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 19:26:19.781860 ip-10-0-128-237 kubenswrapper[2544]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:26:19.781860 ip-10-0-128-237 kubenswrapper[2544]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 19:26:19.781860 ip-10-0-128-237 kubenswrapper[2544]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:26:19.781860 ip-10-0-128-237 kubenswrapper[2544]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 19:26:19.781860 ip-10-0-128-237 kubenswrapper[2544]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 19:26:19.782846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.782711    2544 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 19:26:19.788739 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788724    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:19.788739 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788739    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788743    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788753    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788756    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788759    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788762    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788766    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788769    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788773    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788777    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788781    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788784    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788786    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788789    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788792    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788794    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788797    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788800    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788802    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788806    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:19.788800 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788809    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788813    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788817    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788820    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788823    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788825    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788828    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788831    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788833    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788836    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788838    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788841    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788844    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788846    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788849    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788851    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788854    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788856    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788858    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788861    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:19.789309 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788863    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788866    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788868    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788871    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788873    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788877    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788879    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788882    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788884    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788886    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788889    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788892    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788895    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788898    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788900    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788903    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788905    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788908    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788910    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788913    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:19.789776 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788915    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788918    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788920    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788923    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788926    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788928    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788930    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788933    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788936    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788938    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788941    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788943    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788945    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788948    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788951    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788953    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788955    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788958    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788960    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788963    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:19.790267 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788965    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788968    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788971    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788974    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788977    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788979    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788982    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788984    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788987    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788989    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788993    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.788997    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.789000    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.789003    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.789005    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.789008    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.789011    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.789014    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790264    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:19.790747 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790273    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790277    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790280    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790283    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790301    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790305    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790309    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790311    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790314    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790317    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790319    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790322    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790324    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790327    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790330    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790332    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790335    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790338    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790340    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:19.791186 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790344    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790348    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790350    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790354    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790356    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790359    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790362    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790366    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790368    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790371    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790374    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790377    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790380    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790382    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790384    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790387    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790389    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790392    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790395    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790397    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:19.791651 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790400    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790402    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790405    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790407    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790410    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790412    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790415    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790417    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790420    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790422    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790426    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790429    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790431    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790434    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790437    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790439    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790442    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790444    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790447    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790449    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:19.792149 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790452    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790454    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790457    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790459    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790462    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790464    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790467    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790469    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790472    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790474    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790477    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790479    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790482    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790485    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790488    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790490    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790493    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790496    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790500    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790502    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:19.792648 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790505    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790508    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790510    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790512    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790515    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790519    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790522    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790524    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790527    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790530    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790532    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790535    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790537    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790540    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790542    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790545    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790548    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790550    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790553    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:19.793144 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790894    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790898    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790903    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790906    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790908    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790911    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790914    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790917    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790919    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790922    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790925    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790927    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790929    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790933    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790936    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790939    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790941    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790944    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790946    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790949    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:19.793633 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790951    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790954    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790956    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790959    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790961    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790963    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790965    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790968    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790970    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790973    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790976    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790979    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790981    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790984    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790987    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790990    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790992    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790994    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790997    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.790999    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:19.794122 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791002    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791004    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791007    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791009    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791012    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791014    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791019    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791021    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791024    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791026    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791028    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791031    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791034    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791036    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791039    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791041    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791044    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791046    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791049    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791051    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:19.794605 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791053    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791055    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791059    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791061    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791064    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791066    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791070    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791073    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791075    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791078    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791080    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791083    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791086    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791088    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791091    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791094    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791096    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791098    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791103    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:19.795080 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791107    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791109    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791112    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791114    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791117    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791119    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791123    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791126    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791129    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791132    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791135    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791138    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791140    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791143    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791145    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791147    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791150    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791153    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791155    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791158    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:19.795563 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791224    2544 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791234    2544 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791242    2544 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791248    2544 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791253    2544 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791259    2544 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791264    2544 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791269    2544 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791272    2544 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791275    2544 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791278    2544 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791282    2544 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791298    2544 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791303    2544 flags.go:64] FLAG: --cgroup-root=""
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791307    2544 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791310    2544 flags.go:64] FLAG: --client-ca-file=""
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791312    2544 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791315    2544 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791320    2544 flags.go:64] FLAG: --cluster-domain=""
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791323    2544 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791326    2544 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791329    2544 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791332    2544 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791336    2544 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 19:26:19.796056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791339    2544 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791342    2544 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791345    2544 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791348    2544 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791350    2544 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791353    2544 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791359    2544 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791363    2544 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791367    2544 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791369    2544 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791372    2544 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791375    2544 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791378    2544 flags.go:64] FLAG: --enable-server="true"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791380    2544 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791384    2544 flags.go:64] FLAG: --event-burst="100"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791387    2544 flags.go:64] FLAG: --event-qps="50"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791390    2544 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791393    2544 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791395    2544 flags.go:64] FLAG: --eviction-hard=""
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791399    2544 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791402    2544 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791405    2544 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791408    2544 flags.go:64] FLAG: --eviction-soft=""
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791411    2544 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791414    2544 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 19:26:19.796688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791416    2544 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791419    2544 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791422    2544 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791424    2544 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791427    2544 flags.go:64] FLAG: --feature-gates=""
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791430    2544 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791433    2544 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791437    2544 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791440    2544 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791443    2544 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791446    2544 flags.go:64] FLAG: --help="false"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791449    2544 flags.go:64] FLAG: --hostname-override="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791452    2544 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791455    2544 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791458    2544 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791462    2544 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791465    2544 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791468    2544 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791471    2544 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791474    2544 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791476    2544 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791479    2544 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791482    2544 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791485    2544 flags.go:64] FLAG: --kube-reserved=""
Jul 27 19:26:19.797358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791487    2544 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791490    2544 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791493    2544 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791496    2544 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791498    2544 flags.go:64] FLAG: --lock-file=""
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791501    2544 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791504    2544 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791507    2544 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791512    2544 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791515    2544 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791518    2544 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791520    2544 flags.go:64] FLAG: --logging-format="text"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791523    2544 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791526    2544 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791529    2544 flags.go:64] FLAG: --manifest-url=""
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791531    2544 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791536    2544 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791538    2544 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791542    2544 flags.go:64] FLAG: --max-pods="110"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791545    2544 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791548    2544 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791551    2544 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791554    2544 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791557    2544 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791560    2544 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 19:26:19.797929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791563    2544 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791569    2544 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791572    2544 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791575    2544 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791578    2544 flags.go:64] FLAG: --pod-cidr=""
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791580    2544 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791585    2544 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791587    2544 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791590    2544 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791593    2544 flags.go:64] FLAG: --port="10250"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791596    2544 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791599    2544 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a15b370521eade53"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791602    2544 flags.go:64] FLAG: --qos-reserved=""
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791604    2544 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791607    2544 flags.go:64] FLAG: --register-node="true"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791610    2544 flags.go:64] FLAG: --register-with-taints=""
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791613    2544 flags.go:64] FLAG: --registry-burst="10"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791616    2544 flags.go:64] FLAG: --registry-qps="5"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791618    2544 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791621    2544 flags.go:64] FLAG: --reserved-memory=""
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791625    2544 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791627    2544 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791630    2544 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791633    2544 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791635    2544 flags.go:64] FLAG: --runonce="false"
Jul 27 19:26:19.798529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791638    2544 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791641    2544 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791643    2544 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791646    2544 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791649    2544 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791652    2544 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791655    2544 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791658    2544 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791661    2544 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791664    2544 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791667    2544 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791670    2544 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791673    2544 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791676    2544 flags.go:64] FLAG: --system-cgroups=""
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791681    2544 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791686    2544 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791688    2544 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791691    2544 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791695    2544 flags.go:64] FLAG: --tls-min-version=""
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791698    2544 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791701    2544 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791703    2544 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791706    2544 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791709    2544 flags.go:64] FLAG: --v="2"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791713    2544 flags.go:64] FLAG: --version="false"
Jul 27 19:26:19.799207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791716    2544 flags.go:64] FLAG: --vmodule=""
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791720    2544 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.791723    2544 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791829    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791833    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791836    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791840    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791843    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791845    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791848    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791850    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791853    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791855    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791858    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791861    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791864    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791866    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791869    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791871    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791874    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:19.799814 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791876    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791880    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791882    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791885    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791888    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791890    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791893    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791895    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791897    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791900    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791902    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791905    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791908    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791910    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791912    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791915    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791918    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791920    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791924    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791927    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:19.800405 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791929    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791931    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791934    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791937    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791939    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791941    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791944    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791947    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791949    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791952    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791954    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791957    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791959    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791963    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791966    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791968    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791971    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791973    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791983    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791985    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:19.801166 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791989    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791993    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791996    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.791999    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792002    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792005    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792007    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792010    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792031    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792035    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792039    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792042    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792045    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792047    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792050    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792053    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792056    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792058    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792061    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792063    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:19.801960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792067    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792071    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792075    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792077    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792080    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792083    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792086    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792088    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792090    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792093    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792096    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792098    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792100    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792103    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792105    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792108    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792110    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792113    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792115    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:19.802573 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792118    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792120    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.792123    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.792130    2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.798641    2544 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.798653    2544 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.798688    2544 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.798698    2544 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798756    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798761    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798763    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798766    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798769    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798773    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798776    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:19.803065 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798779    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798782    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798784    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798787    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798789    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798792    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798795    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798797    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798800    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798803    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798805    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798808    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798810    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798813    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798815    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798818    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798820    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798823    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798825    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798827    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:19.803448 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798830    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798833    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798836    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798838    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798841    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798843    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798846    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798848    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798851    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798853    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798856    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798859    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798861    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798864    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798867    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798869    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798871    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798874    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798876    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798879    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:19.803912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798881    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798883    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798886    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798888    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798890    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798893    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798895    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798898    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798900    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798902    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798905    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798907    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798910    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798912    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798917    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798921    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798923    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798926    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798928    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:19.804447 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798932    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798934    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798936    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798939    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798941    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798944    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798947    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798949    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798952    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798954    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798957    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798959    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798961    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798964    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798966    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798969    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798971    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798974    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798978    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798981    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:19.804902 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798984    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798986    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798989    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798991    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798994    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.798997    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799000    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799003    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799006    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799008    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799011    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799014    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799016    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.799021    2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:26:19.805397 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799148    2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799152    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799155    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799159    2544 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799162    2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799165    2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799167    2544 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799170    2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799172    2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799175    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799177    2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799179    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799182    2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799184    2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799186    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799189    2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799191    2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799195    2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799198    2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799201    2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 19:26:19.805729 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799204    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799207    2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799210    2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799212    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799215    2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799218    2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799220    2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799223    2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799225    2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799228    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799230    2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799233    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799235    2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799237    2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799240    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799242    2544 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799245    2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799248    2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799250    2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799252    2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 19:26:19.806200 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799255    2544 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799258    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799260    2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799263    2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799265    2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799267    2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799270    2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799272    2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799274    2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799277    2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799279    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799282    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799284    2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799300    2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799303    2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799306    2544 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799309    2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799311    2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799314    2544 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799316    2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 19:26:19.806694 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799318    2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799321    2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799324    2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799326    2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799328    2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799331    2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799333    2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799336    2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799338    2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799340    2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799344    2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799346    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799349    2544 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799351    2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799353    2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799356    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799358    2544 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799360    2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799363    2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799367    2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 19:26:19.807177 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799371    2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799374    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799377    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799380    2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799382    2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799385    2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799387    2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799390    2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799394    2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799397    2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799399    2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799402    2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799405    2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799408    2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799410    2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799413    2544 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799416    2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799418    2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 19:26:19.807766 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:19.799421    2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.799425    2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.799523    2544 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.802770    2544 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.804051    2544 server.go:1019] "Starting client certificate rotation"
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.804149    2544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.804178    2544 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.806169    2544 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:26:19.808208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.807260    2544 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 19:26:19.817271 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.817256    2544 log.go:25] "Validated CRI v1 runtime API"
Jul 27 19:26:19.819100 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.819087    2544 log.go:25] "Validated CRI v1 image API"
Jul 27 19:26:19.820589 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.820572    2544 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 19:26:19.822719 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.822690    2544 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cfcb5b22-c45b-4d8f-be9d-506e54bdce3c:/dev/nvme0n1p4 d709ad59-f39f-48ac-8a83-05209b61825a:/dev/nvme0n1p3]
Jul 27 19:26:19.822804 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.822716    2544 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 19:26:19.828479 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.828361    2544 manager.go:216] Machine: {Timestamp:2026-07-27 19:26:19.827196296 +0000 UTC m=+0.309347886 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3202801 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec285fbe1e438872aaa266f7a78399f9 SystemUUID:ec285fbe-1e43-8872-aaa2-66f7a78399f9 BootID:5d319369-5ed9-415e-893d-a1f145fc5747 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0b:b3:86:49:bf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0b:b3:86:49:bf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:89:9d:cb:5d:a2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 19:26:19.828479 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.828472    2544 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 19:26:19.828636 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.828569    2544 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 19:26:19.829428 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.829406    2544 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 19:26:19.829581 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.829430    2544 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-237.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 19:26:19.830110 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.830099    2544 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 19:26:19.830159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.830116    2544 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 19:26:19.830159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.830127    2544 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:26:19.830779 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.830768    2544 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 19:26:19.831991 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.831970    2544 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 19:26:19.834759 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834746    2544 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:26:19.834920 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834909    2544 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 19:26:19.834983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834940    2544 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 19:26:19.834983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834941    2544 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:26:19.834983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834954    2544 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 19:26:19.834983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834970    2544 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 19:26:19.834983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834982    2544 kubelet.go:398] "Adding apiserver pod source"
Jul 27 19:26:19.835170 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.834993    2544 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 19:26:19.835365 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.835352    2544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:26:19.835435 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.835375    2544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 19:26:19.837496 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.837479    2544 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 19:26:19.840945 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.840926    2544 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 19:26:19.841016 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.840969    2544 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 19:26:19.842462 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842448    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 19:26:19.842495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842466    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 19:26:19.842495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842473    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 19:26:19.842495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842480    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 19:26:19.842495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842492    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842501    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842509    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842514    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842521    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842527    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842535    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 19:26:19.842598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.842541    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 19:26:19.843879 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.843866    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 19:26:19.843918 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.843881    2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 19:26:19.845675 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.845651    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:19.845732 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.845651    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:19.845815 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.845798    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:19.847164 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.847153    2544 server.go:1268] "Started kubelet"
Jul 27 19:26:19.847271 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.847224    2544 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 19:26:19.847414 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.847302    2544 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 19:26:19.847414 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.847241    2544 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 19:26:19.847915 ip-10-0-128-237 systemd[1]: Started Kubernetes Kubelet.
Jul 27 19:26:19.850708 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.850692    2544 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 19:26:19.851213 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.851197    2544 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 19:26:19.852964 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.852947    2544 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 19:26:19.856616 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.856596    2544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 19:26:19.857551 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.857537    2544 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 19:26:19.857657 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.857642    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:26:19.858061 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858045    2544 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 19:26:19.858061 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858048    2544 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 19:26:19.858190 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858069    2544 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 19:26:19.858190 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858183    2544 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 19:26:19.858279 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858192    2544 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 19:26:19.858335 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858308    2544 factory.go:55] Registering systemd factory
Jul 27 19:26:19.858335 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858324    2544 factory.go:223] Registration of the systemd container factory successfully
Jul 27 19:26:19.858541 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858520    2544 factory.go:153] Registering CRI-O factory
Jul 27 19:26:19.858577 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858543    2544 factory.go:223] Registration of the crio container factory successfully
Jul 27 19:26:19.858607 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858581    2544 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 19:26:19.858607 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858604    2544 factory.go:103] Registering Raw factory
Jul 27 19:26:19.858673 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.858618    2544 manager.go:1196] Started watching for new ooms in manager
Jul 27 19:26:19.859062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.859050    2544 manager.go:349] Starting recovery of all containers
Jul 27 19:26:19.862329 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.862283    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:19.862541 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.862502    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 27 19:26:19.863254 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.862329    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e0c8f27d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.847127677 +0000 UTC m=+0.329279267,LastTimestamp:2026-07-27 19:26:19.847127677 +0000 UTC m=+0.329279267,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.870371 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.870350    2544 manager.go:354] Recovery completed
Jul 27 19:26:19.874911 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.874898    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:19.876584 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.876566    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:19.876658 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.876602    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:19.876658 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.876617    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:19.877504 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.877345    2544 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 19:26:19.877504 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.877358    2544 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 19:26:19.877504 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.877375    2544 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 19:26:19.877824 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.877811    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:19.878658 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.878641    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:19.878725 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.878667    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:19.878725 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.878678    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:19.878725 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.878628    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.879228 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.879213    2544 policy_none.go:49] "None policy: Start"
Jul 27 19:26:19.879321 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.879235    2544 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 19:26:19.879321 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.879248    2544 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 19:26:19.880315 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.880304    2544 policy_none.go:47] "Start"
Jul 27 19:26:19.890599 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.890330    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.901979 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.901911    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.913802 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.912780    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:19.878655205 +0000 UTC m=+0.360806797,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.922354 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922341    2544 manager.go:337] "Starting Device Plugin manager"
Jul 27 19:26:19.922423 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.922368    2544 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 19:26:19.922423 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922378    2544 server.go:87] "Starting device plugin registration server"
Jul 27 19:26:19.922620 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922594    2544 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 19:26:19.922620 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922608    2544 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 19:26:19.922737 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922678    2544 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 19:26:19.922802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922781    2544 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 19:26:19.922802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:19.922794    2544 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 19:26:19.923241 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.923206    2544 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 19:26:19.923328 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.923260    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:26:19.925203 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.925148    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:19.878672088 +0000 UTC m=+0.360823678,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.936880 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.936803    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28b0041\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:19.878682854 +0000 UTC m=+0.360834445,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:19.950079 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:19.950018    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e6102328  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.935679272 +0000 UTC m=+0.417830854,LastTimestamp:2026-07-27 19:26:19.935679272 +0000 UTC m=+0.417830854,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.008169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.008136    2544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 19:26:20.009475 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.009459    2544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 19:26:20.009559 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.009492    2544 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 19:26:20.009559 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.009512    2544 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 19:26:20.009559 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.009541    2544 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 19:26:20.017761 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.017736    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:20.022811 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.022797    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:20.023460 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.023436    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:20.023541 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.023466    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:20.023541 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.023477    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:20.023541 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.023497    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.032301 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.032190    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:20.023451082 +0000 UTC m=+0.505602672,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.041721 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.041701    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.041800 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.041744    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:20.023472719 +0000 UTC m=+0.505624308,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.051048 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.050994    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28b0041\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:20.023481723 +0000 UTC m=+0.505633314,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.075478 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.075459    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Jul 27 19:26:20.109633 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.109608    2544 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"]
Jul 27 19:26:20.109696 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.109670    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:20.110495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.110482    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:20.110547 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.110506    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:20.110547 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.110516    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:20.111826 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.111813    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:20.111947 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.111933    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.111992 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.111966    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:20.112529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.112513    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:20.112621 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.112540    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:20.112621 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.112553    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:20.112621 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.112514    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:20.112621 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.112617    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:20.112813 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.112629    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:20.114037 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.114020    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.114105 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.114051    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:20.114675 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.114661    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:20.114754 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.114680    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:20.114754 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.114702    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:20.119465 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.119407    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:20.110494629 +0000 UTC m=+0.592646222,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.128422 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.128366    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:20.110511237 +0000 UTC m=+0.592662829,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.137053 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.136992    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28b0041\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:20.11051987 +0000 UTC m=+0.592671461,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.138777 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.138761    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.143131 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.143116    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.148135 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.148060    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:20.112530445 +0000 UTC m=+0.594682044,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.156642 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.156578    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:20.112546449 +0000 UTC m=+0.594698044,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.159945 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.159924    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91a72cf2fcdcd15b4bb9ab9ab54878da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"91a72cf2fcdcd15b4bb9ab9ab54878da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.160056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.159963    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91a72cf2fcdcd15b4bb9ab9ab54878da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"91a72cf2fcdcd15b4bb9ab9ab54878da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.160056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:20.159986    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b923290a23099a90256d2af76020e79-config\") pod \"kube-apiserver-proxy-ip-10-0-128-237.ec2.internal\" (UID: \"3b923290a23099a90256d2af76020e79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:20.165843 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.165785    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28b0041\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:20.112558563 +0000 UTC m=+0.594710156,Count:5,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.173827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.173774    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:20.112601468 +0000 UTC m=+0.594753065,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.183598 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.183546    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:20.112624663 +0000 UTC m=+0.594776254,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:20.192096 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:20.191998    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28b0041\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:20.112633566 +0000 UTC m=+0.594785157,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.025019 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.024963    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:20.114673281 +0000 UTC m=+0.596824871,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.032764 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.032692    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:20.114686412 +0000 UTC m=+0.596838003,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.042206 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.042150    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28b0041\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28b0041  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876622401 +0000 UTC m=+0.358773993,LastTimestamp:2026-07-27 19:26:20.114707349 +0000 UTC m=+0.596858940,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.066207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.066187    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:22.066950 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.066936    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:22.067005 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.066964    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:22.067005 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.066974    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:22.067005 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.066995    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.075552 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.075490    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28a7054\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28a7054  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876585556 +0000 UTC m=+0.358737152,LastTimestamp:2026-07-27 19:26:22.066950744 +0000 UTC m=+0.724660265,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.083982 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.083964    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.084092 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.084039    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18c63c18e28acb38\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18c63c18e28acb38  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:19.876608824 +0000 UTC m=+0.358760419,LastTimestamp:2026-07-27 19:26:22.066969254 +0000 UTC m=+0.724678774,Count:8,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.085117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.085101    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91a72cf2fcdcd15b4bb9ab9ab54878da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"91a72cf2fcdcd15b4bb9ab9ab54878da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.085182 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.085130    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91a72cf2fcdcd15b4bb9ab9ab54878da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"91a72cf2fcdcd15b4bb9ab9ab54878da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.085182 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.085153    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b923290a23099a90256d2af76020e79-config\") pod \"kube-apiserver-proxy-ip-10-0-128-237.ec2.internal\" (UID: \"3b923290a23099a90256d2af76020e79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.085247 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.085213    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91a72cf2fcdcd15b4bb9ab9ab54878da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"91a72cf2fcdcd15b4bb9ab9ab54878da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.085288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.085272    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91a72cf2fcdcd15b4bb9ab9ab54878da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"91a72cf2fcdcd15b4bb9ab9ab54878da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.085326 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.085317    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3b923290a23099a90256d2af76020e79-config\") pod \"kube-apiserver-proxy-ip-10-0-128-237.ec2.internal\" (UID: \"3b923290a23099a90256d2af76020e79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.267886 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.267831    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.270319 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.270297    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.309417 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.309396    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Jul 27 19:26:22.484085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.484051    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:22.485805 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.485788    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:22.485876 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.485819    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:22.485876 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.485829    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:22.485876 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.485856    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.505666 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.505634    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:22.654872 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.654796    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:22.681644 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.681616    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:22.841594 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:22.841562    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b923290a23099a90256d2af76020e79.slice/crio-a5d77d3673d8ac717ad9c2141f22454e20ba09d30b5e698141d2210f8bd282e9 WatchSource:0}: Error finding container a5d77d3673d8ac717ad9c2141f22454e20ba09d30b5e698141d2210f8bd282e9: Status 404 returned error can't find the container with id a5d77d3673d8ac717ad9c2141f22454e20ba09d30b5e698141d2210f8bd282e9
Jul 27 19:26:22.842170 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:26:22.842146    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a72cf2fcdcd15b4bb9ab9ab54878da.slice/crio-581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 WatchSource:0}: Error finding container 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657: Status 404 returned error can't find the container with id 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657
Jul 27 19:26:22.848451 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.848203    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:26:22.857392 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.857324    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1993adff8c  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\",Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:22.848483212 +0000 UTC m=+1.506192721,LastTimestamp:2026-07-27 19:26:22.848483212 +0000 UTC m=+1.506192721,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.869070 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:22.869003    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-128-237.ec2.internal.18c63c1993b5a908  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-128-237.ec2.internal,UID:3b923290a23099a90256d2af76020e79,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulling,Message:Pulling image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\",Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:22.848985352 +0000 UTC m=+1.506694859,LastTimestamp:2026-07-27 19:26:22.848985352 +0000 UTC m=+1.506694859,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:22.914476 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.914424    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:22.915404 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.915390    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:22.915487 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.915420    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:22.915487 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:22.915434    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:23.121320 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:23.121289    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="1.6s"
Jul 27 19:26:23.121424 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:23.121334    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:23.270607 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:23.270533    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:23.305989 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.305961    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:23.307004 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.306982    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:23.307109 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.307028    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:23.307109 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.307045    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:23.307109 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.307077    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:23.327383 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:23.327356    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:23.413186 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:23.413146    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:23.684863 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.684784    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:23.839184 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.839125    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerStarted","Data":"581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657"}
Jul 27 19:26:23.840316 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:23.840280    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal" event={"ID":"3b923290a23099a90256d2af76020e79","Type":"ContainerStarted","Data":"a5d77d3673d8ac717ad9c2141f22454e20ba09d30b5e698141d2210f8bd282e9"}
Jul 27 19:26:24.613221 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.613141    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c19fc4da4fc  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" in 1.755s (1.755s including waiting). Image size: 278280461 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.603776252 +0000 UTC m=+3.261485759,LastTimestamp:2026-07-27 19:26:24.603776252 +0000 UTC m=+3.261485759,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:24.633010 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.632927    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-128-237.ec2.internal.18c63c19fd87f24f  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-128-237.ec2.internal,UID:3b923290a23099a90256d2af76020e79,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Pulled,Message:Successfully pulled image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559\" in 1.775s (1.775s including waiting). Image size: 317186379 bytes.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.624374351 +0000 UTC m=+3.282083880,LastTimestamp:2026-07-27 19:26:24.624374351 +0000 UTC m=+3.282083880,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:24.678988 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.678962    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:24.696553 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.696459    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-128-237.ec2.internal.18c63c1a012b6fe1  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-128-237.ec2.internal,UID:3b923290a23099a90256d2af76020e79,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Created,Message:Created container: haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.685420513 +0000 UTC m=+3.343130022,LastTimestamp:2026-07-27 19:26:24.685420513 +0000 UTC m=+3.343130022,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:24.705088 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.705002    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"kube-system\"" event="&Event{ObjectMeta:{kube-apiserver-proxy-ip-10-0-128-237.ec2.internal.18c63c1a0194bb78  kube-system    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:kube-system,Name:kube-apiserver-proxy-ip-10-0-128-237.ec2.internal,UID:3b923290a23099a90256d2af76020e79,APIVersion:v1,ResourceVersion:,FieldPath:spec.containers{haproxy},},Reason:Started,Message:Started container haproxy,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.692321144 +0000 UTC m=+3.350030664,LastTimestamp:2026-07-27 19:26:24.692321144 +0000 UTC m=+3.350030664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:24.730703 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.730680    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="3.2s"
Jul 27 19:26:24.806005 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.805935    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a07d77998  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.797358488 +0000 UTC m=+3.455067996,LastTimestamp:2026-07-27 19:26:24.797358488 +0000 UTC m=+3.455067996,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:24.814145 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.814084    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a083fde81  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.804200065 +0000 UTC m=+3.461909572,LastTimestamp:2026-07-27 19:26:24.804200065 +0000 UTC m=+3.461909572,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:24.845126 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.845093    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerStarted","Data":"6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3"}
Jul 27 19:26:24.845228 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.845131    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:24.846453 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.846430    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal" event={"ID":"3b923290a23099a90256d2af76020e79","Type":"ContainerStarted","Data":"bd4b8da09e64634793fcf04357f2e1880427e9d2459cf38f71c88c6f61893bb2"}
Jul 27 19:26:24.846529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.846474    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:24.848983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.848962    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:24.848983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.848972    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:24.849121 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.848996    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:24.849121 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.848999    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:24.849121 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.849007    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:24.849121 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.849013    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:24.849258 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.849168    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:24.849291 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.849258    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:24.927807 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.927756    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:24.928574 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.928561    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:24.928630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.928586    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:24.928630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.928596    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:24.928630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:24.928617    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:24.945997 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.945964    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:24.946076 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:24.946021    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:25.284482 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:25.284403    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:25.314505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.314489    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:25.315257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.315241    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:25.315319 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.315271    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:25.315319 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.315281    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:25.318554 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:25.318534    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:25.673642 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.673591    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:25.768317 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:25.768293    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:25.847810 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.847783    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:25.847903 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.847788    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:25.848991 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.848974    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:25.849087 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.849010    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:25.849087 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.849023    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:25.849087 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.848977    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:25.849087 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.849085    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:25.849271 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:25.849096    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:25.849320 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:25.849273    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:25.849371 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:25.849337    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:26.688257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:26.688076    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:27.679366 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:27.679330    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:27.942447 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:27.942384    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="6.4s"
Jul 27 19:26:28.146275 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:28.146257    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:28.147061 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:28.147040    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:28.147144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:28.147074    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:28.147144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:28.147083    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:28.147144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:28.147109    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:28.163437 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:28.163416    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:28.679921 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:28.679895    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:29.174257 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:29.174179    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:29.421200 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:29.421174    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:29.422630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:29.422609    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:29.422723 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:29.422641    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:29.422723 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:29.422650    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:29.681811 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:29.681792    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:30.234655 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:30.234630    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:30.679069 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:30.678997    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:31.037655 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:31.037583    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:31.658923 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:31.658897    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:31.679098 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:31.679074    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:31.747819 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:31.747776    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:26:32.680240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:32.680215    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:33.679915 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:33.679892    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:34.352253 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:34.352229    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:34.564271 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:34.564252    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:34.565194 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:34.565173    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:34.565291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:34.565205    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:34.565291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:34.565217    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:34.565291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:34.565245    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:34.582710 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:34.582684    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:34.679001 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:34.678943    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:35.683851 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:35.683825    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:36.682782 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.682759    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:36.865586 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.865560    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/0.log"
Jul 27 19:26:36.865917 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.865600    2544 generic.go:356] "Generic (PLEG): container finished" podID="91a72cf2fcdcd15b4bb9ab9ab54878da" containerID="6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3" exitCode=1
Jul 27 19:26:36.865917 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.865631    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerDied","Data":"6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3"}
Jul 27 19:26:36.865917 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.865802    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:36.866490 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.866474    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:36.866573 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.866502    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:36.866573 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:36.866514    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:36.866721 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:36.866706    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:36.878001 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:36.877900    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1cd7687109  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:36.869677321 +0000 UTC m=+15.527386834,LastTimestamp:2026-07-27 19:26:36.869677321 +0000 UTC m=+15.527386834,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:36.948892 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:36.948822    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a07d77998\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a07d77998  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.797358488 +0000 UTC m=+3.455067996,LastTimestamp:2026-07-27 19:26:36.939057217 +0000 UTC m=+15.596766728,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:36.956214 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:36.956149    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a083fde81\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a083fde81  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.804200065 +0000 UTC m=+3.461909572,LastTimestamp:2026-07-27 19:26:36.948597522 +0000 UTC m=+15.606307044,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:37.657956 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:37.657916    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:37.681564 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.681538    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:37.869169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.869149    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/0.log"
Jul 27 19:26:37.869483 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.869191    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerStarted","Data":"b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22"}
Jul 27 19:26:37.869483 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.869312    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:37.870693 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.870672    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:37.870785 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.870704    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:37.870785 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:37.870714    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:37.870902 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:37.870890    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:38.467168 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:38.467144    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:38.468004 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:38.467990    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:38.468079 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:38.468023    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:38.468079 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:38.468034    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:38.682000 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:38.681971    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:39.682273 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:39.682246    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:40.577520 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:40.577490    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:40.679953 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:40.679925    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:41.361916 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:41.361882    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:41.583392 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:41.583354    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:41.584360 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:41.584345    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:41.584424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:41.584374    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:41.584424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:41.584384    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:41.584424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:41.584411    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:41.605238 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:41.605208    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:41.679528 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:41.679464    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:41.748979 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:41.748943    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:26:41.764516 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:41.764488    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:26:42.435858 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:42.435823    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:42.679223 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:42.679193    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:43.680182 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:43.680150    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:44.679543 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:44.679510    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:45.680939 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:45.680909    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:46.678807 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:46.678780    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:47.680166 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:47.680137    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:48.372007 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.371933    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:48.605947 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.605924    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:48.606904 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.606881    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:48.607018 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.606921    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:48.607018 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.606932    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:48.607018 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.606956    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:48.625176 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.625112    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:48.678390 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.678372    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:48.886985 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.886937    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/1.log"
Jul 27 19:26:48.887288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.887247    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/0.log"
Jul 27 19:26:48.887288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.887274    2544 generic.go:356] "Generic (PLEG): container finished" podID="91a72cf2fcdcd15b4bb9ab9ab54878da" containerID="b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22" exitCode=1
Jul 27 19:26:48.887366 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.887300    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerDied","Data":"b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22"}
Jul 27 19:26:48.887366 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.887338    2544 scope.go:117] "RemoveContainer" containerID="6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3"
Jul 27 19:26:48.887462 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.887448    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:48.888208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.888187    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:48.888284 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.888223    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:48.888284 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.888236    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:48.888616 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.888489    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:48.888616 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:48.888553    2544 scope.go:117] "RemoveContainer" containerID="6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3"
Jul 27 19:26:48.896912 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.896879    2544 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator_91a72cf2fcdcd15b4bb9ab9ab54878da_0 in pod sandbox 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 from index: no such id: '6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3'" containerID="6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3"
Jul 27 19:26:48.896989 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.896926    2544 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator_91a72cf2fcdcd15b4bb9ab9ab54878da_0 in pod sandbox 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 from index: no such id: '6476973b35986041cf73e606cdd16ae4315dec81e938ada2e5c3f110e74127a3'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da)\"" logger="UnhandledError"
Jul 27 19:26:48.897073 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.897051    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" podUID="91a72cf2fcdcd15b4bb9ab9ab54878da"
Jul 27 19:26:48.904789 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:48.904697    2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1fa44b1ce5  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da),Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:48.897019109 +0000 UTC m=+27.554728631,LastTimestamp:2026-07-27 19:26:48.897019109 +0000 UTC m=+27.554728631,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:49.679685 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:49.679660    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:49.890218 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:49.890195    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/1.log"
Jul 27 19:26:50.679214 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:50.679188    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:51.681144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:51.681114    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:51.750016 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:51.749991    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:26:52.679944 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:52.679919    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:53.492089 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:53.492053    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 19:26:53.679202 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:53.679176    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:54.686139 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:54.686113    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:55.381749 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:55.381696    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:26:55.626197 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:55.626167    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:55.627108 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:55.627091    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:55.627194 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:55.627123    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:55.627194 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:55.627133    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:55.627194 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:55.627163    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:55.642645 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:55.642594    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:55.679296 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:55.679277    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:56.417519 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:56.417488    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:56.418545 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:56.418530    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:56.418609 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:56.418560    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:56.418609 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:56.418570    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:56.678877 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:56.678802    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:57.211277 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:57.211250    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 19:26:57.660635 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:57.660606    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Jul 27 19:26:57.678443 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:57.678419    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:58.681415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:58.681380    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:59.679457 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:59.679424    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:26:59.834547 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:59.834503    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:26:59.835478 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:59.835458    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:26:59.835574 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:59.835494    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:26:59.835574 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:26:59.835509    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:26:59.835758 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:59.835726    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:26:59.846166 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:59.846074    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1cd7687109\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1cd7687109  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Pulled,Message:Container image \"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\" already present on machine,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:36.869677321 +0000 UTC m=+15.527386834,LastTimestamp:2026-07-27 19:26:59.836536448 +0000 UTC m=+38.494245960,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:59.916004 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:59.915923    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a07d77998\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a07d77998  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Created,Message:Created container: setup,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.797358488 +0000 UTC m=+3.455067996,LastTimestamp:2026-07-27 19:26:59.907308362 +0000 UTC m=+38.565017890,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:26:59.924198 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:26:59.924112    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a083fde81\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1a083fde81  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:Started,Message:Started container setup,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:24.804200065 +0000 UTC m=+3.461909572,LastTimestamp:2026-07-27 19:26:59.915570289 +0000 UTC m=+38.573279822,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:27:00.681159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.681127    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:00.908498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.908477    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/1.log"
Jul 27 19:27:00.908857 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.908527    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerStarted","Data":"b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b"}
Jul 27 19:27:00.908857 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.908645    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:00.909553 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.909538    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:00.909612 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.909569    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:00.909612 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:00.909582    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:00.909805 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:00.909792    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:01.680183 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:01.680148    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:01.750274 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:01.750243    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:02.389982 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:02.389951    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:27:02.643763 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:02.643670    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:02.644693 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:02.644674    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:02.644792 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:02.644710    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:02.644792 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:02.644721    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:02.644792 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:02.644768    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:02.663086 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:02.663054    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:02.679586 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:02.679554    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:03.184188 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:03.184163    2544 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 27 19:27:03.680809 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:03.680776    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:04.678673 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:04.678637    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:05.681201 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:05.681165    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:06.680250 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:06.680221    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:07.679934 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:07.679910    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:08.681018 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:08.680996    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:09.400686 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:09.400650    2544 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="7s"
Jul 27 19:27:09.664153 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:09.664068    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:09.666878 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:09.666851    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:09.667036 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:09.666891    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:09.667036 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:09.666906    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:09.667036 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:09.666938    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:09.674792 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:09.674769    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:09.678494 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:09.678469    2544 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:10.673521 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:10.673493    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:11.679006 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.678975    2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 19:27:11.750834 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:11.750782    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:11.925606 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.925581    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/2.log"
Jul 27 19:27:11.925972 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.925958    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/1.log"
Jul 27 19:27:11.926035 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.925990    2544 generic.go:356] "Generic (PLEG): container finished" podID="91a72cf2fcdcd15b4bb9ab9ab54878da" containerID="b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b" exitCode=1
Jul 27 19:27:11.926035 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.926015    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerDied","Data":"b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b"}
Jul 27 19:27:11.926097 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.926082    2544 scope.go:117] "RemoveContainer" containerID="b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22"
Jul 27 19:27:11.926156 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.926146    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:11.927247 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.927085    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:11.927247 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.927110    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:11.927247 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.927120    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:11.927361 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:11.927301    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:11.927361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.927347    2544 scope.go:117] "RemoveContainer" containerID="b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22"
Jul 27 19:27:11.935259 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:11.935233    2544 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator_91a72cf2fcdcd15b4bb9ab9ab54878da_1 in pod sandbox 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 from index: no such id: 'b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22'" containerID="b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22"
Jul 27 19:27:11.935362 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:11.935273    2544 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator_91a72cf2fcdcd15b4bb9ab9ab54878da_1 in pod sandbox 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 from index: no such id: 'b65336d7ca8d754370a5438e2d02a5e69ec8ebe460a0b54e5f9dd3574d142a22'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da)\"" logger="UnhandledError"
Jul 27 19:27:11.935421 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:11.935402    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" podUID="91a72cf2fcdcd15b4bb9ab9ab54878da"
Jul 27 19:27:11.944382 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:11.944309    2544 event.go:359] "Server rejected event (will not retry!)" err="events \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1fa44b1ce5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"openshift-machine-config-operator\"" event="&Event{ObjectMeta:{kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal.18c63c1fa44b1ce5  openshift-machine-config-operator    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-machine-config-operator,Name:kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal,UID:91a72cf2fcdcd15b4bb9ab9ab54878da,APIVersion:v1,ResourceVersion:,FieldPath:spec.initContainers{setup},},Reason:BackOff,Message:Back-off restarting failed container setup in pod kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da),Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-07-27 19:26:48.897019109 +0000 UTC m=+27.554728631,LastTimestamp:2026-07-27 19:27:11.935369616 +0000 UTC m=+50.593079123,Count:2,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Jul 27 19:27:11.995777 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:11.995743    2544 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-smmgx"
Jul 27 19:27:12.003920 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:12.003900    2544 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-smmgx"
Jul 27 19:27:12.628349 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:12.628319    2544 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 19:27:12.685012 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:12.684987    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:12.701500 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:12.701479    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:12.759476 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:12.759451    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:12.928774 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:12.928692    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/2.log"
Jul 27 19:27:13.008112 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.008064    2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 19:22:11 +0000 UTC" deadline="2028-01-28 16:30:44.851546214 +0000 UTC"
Jul 27 19:27:13.008112 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.008110    2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13197h3m31.843438922s"
Jul 27 19:27:13.028329 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.028309    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.028380 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:13.028330    2544 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.064696 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.064673    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.080671 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.080651    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.141510 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.141492    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.407363 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.407335    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.407363 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:13.407359    2544 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.662435 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.662360    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.678156 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.678132    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.734005 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.733969    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.996065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:13.995988    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:13.996065 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:13.996015    2544 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:15.936010 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:15.935979    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:15.951806 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:15.951786    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:16.013802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.013781    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:16.284508 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.284428    2544 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:16.284508 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:16.284474    2544 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-128-237.ec2.internal" not found
Jul 27 19:27:16.406423 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:16.406392    2544 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:16.678848 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.678812    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:16.685335 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.685317    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:16.685402 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.685353    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:16.685402 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.685367    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:16.685402 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.685393    2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:16.695677 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:16.695659    2544 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:16.695725 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:16.695686    2544 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-237.ec2.internal\": node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:16.710906 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:16.710885    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:16.812011 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:16.811979    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:16.912204 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:16.912176    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.012825 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.012722    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.113521 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.113489    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.214339 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.214305    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.314840 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.314780    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.415057 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.415017    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.515571 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.515545    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.616044 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.616020    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.716244 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.716214    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.816874 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.816838    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:17.918010 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:17.917923    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.018877 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.018848    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.119362 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.119322    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.219642 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.219569    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.320513 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.320482    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.421519 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.421490    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.522075 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.522001    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.622819 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.622784    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.723240 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.723205    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.823899 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.823835    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:18.924724 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:18.924688    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.025167 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.025128    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.126006 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.125968    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.226717 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.226681    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.327174 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.327150    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.427760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.427685    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.528189 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.528160    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.628757 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.628712    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.728989 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.728915    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.830031 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.830001    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:19.930627 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:19.930589    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.031033 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.030945    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.131385 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.131352    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.231958 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.231923    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.332322 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.332250    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.432667 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.432627    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.533086 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.533044    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.634024 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.633997    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.735035 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.734995    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.835105 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.835081    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:20.935658 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:20.935591    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.036038 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.035995    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.136629 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.136594    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.237766 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.237673    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.338167 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.338137    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.438673 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.438648    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.539149 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.539096    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.639545 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.639515    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.740029 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.739986    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.751324 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.751300    2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.840904 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.840830    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:21.941349 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:21.941305    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.041790 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.041725    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.142380 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.142345    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.243021 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.242983    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.343471 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.343449    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.443972 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.443910    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.544407 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.544377    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.645016 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.644990    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.746139 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.746071    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.834155 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:22.834131    2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 19:27:22.835923 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:22.835904    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 19:27:22.836034 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:22.835938    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 19:27:22.836034 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:22.835951    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Jul 27 19:27:22.836189 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.836175    2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Jul 27 19:27:22.836352 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.836334    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" podUID="91a72cf2fcdcd15b4bb9ab9ab54878da"
Jul 27 19:27:22.846974 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.846952    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:22.947408 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:22.947380    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:23.047884 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.047817    2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Jul 27 19:27:23.073449 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.073429    2544 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:23.082442 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.082412    2544 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Jul 27 19:27:23.098561 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.098536    2544 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:27:23.099828 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.099813    2544 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Jul 27 19:27:23.107685 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.107672    2544 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 19:27:23.692870 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.692839    2544 apiserver.go:52] "Watching apiserver"
Jul 27 19:27:23.699071 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.699040    2544 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 19:27:23.699416 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.699398    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-pw577","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal","openshift-multus/multus-qqs98","openshift-network-diagnostics/network-check-target-2lglj","openshift-network-operator/iptables-alerter-s6fhr","openshift-ovn-kubernetes/ovnkube-node-mvvlt","kube-system/global-pull-secret-syncer-jks2p","kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4","openshift-cluster-node-tuning-operator/tuned-8dcff","openshift-dns/node-resolver-757g8","openshift-image-registry/node-ca-xg9lr","openshift-multus/multus-additional-cni-plugins-xk5wh","openshift-multus/network-metrics-daemon-7d4b9"]
Jul 27 19:27:23.701558 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.701540    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.701627 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.701610    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:23.703358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.703341    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.705272 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.705246    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.706044 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.706016    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 19:27:23.707708 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.706988    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.707708 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.707219    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 19:27:23.707708 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.707333    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.707909 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.707865    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 19:27:23.708365 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.708342    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:23.709336 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.709210    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:23.709454 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.709384    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xng96\""
Jul 27 19:27:23.709454 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.709392    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f2lp5\""
Jul 27 19:27:23.709575 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.709455    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 19:27:23.711252 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.711232    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.713333 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.713313    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 19:27:23.713427 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.713334    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.713427 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.713320    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.713580 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.713564    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.713640 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.713606    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-48rxj\""
Jul 27 19:27:23.714657 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.714642    2544 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 19:27:23.715645 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.715633    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.715822 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.715722    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 19:27:23.715984 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.715965    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 19:27:23.715984 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.716004    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 19:27:23.716133 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.716024    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pklxz\""
Jul 27 19:27:23.716218 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.716197    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.716272 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.716219    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.716360 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.716318    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 19:27:23.717802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.717787    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.718329 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.718307    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.718329 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.718311    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 19:27:23.718466 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.718362    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dvtqg\""
Jul 27 19:27:23.718466 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.718313    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.720084 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.720070    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:23.720155 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.720130    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:23.720224 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.720166    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 19:27:23.720403 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.720380    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 19:27:23.720548 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.720532    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6zvbh\""
Jul 27 19:27:23.722466 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.722453    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.724834 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.724815    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.727161 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.727147    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.727278 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.727252    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"setup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=setup pod=kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator(91a72cf2fcdcd15b4bb9ab9ab54878da)\"" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" podUID="91a72cf2fcdcd15b4bb9ab9ab54878da"
Jul 27 19:27:23.728853 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.728818    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.728936 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.728893    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.728985 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.728961    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nmhfp\""
Jul 27 19:27:23.729206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729188    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lrdkg\""
Jul 27 19:27:23.729275 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729214    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.729275 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729191    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.729784 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729719    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 19:27:23.729784 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729775    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f4fwn\""
Jul 27 19:27:23.729907 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729803    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 19:27:23.729907 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.729813    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 19:27:23.730825 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.730799    2544 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 19:27:23.776723 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.776701    2544 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qxbxm"
Jul 27 19:27:23.783527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.783503    2544 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 19:27:23.785830 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.785812    2544 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qxbxm"
Jul 27 19:27:23.803538 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803497    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-etc-kubernetes\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.803538 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803539    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysctl-d\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.803746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803556    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-host\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.803746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803574    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-tuned\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.803746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803623    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-cni-bin\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.803746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803654    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-ovnkube-config\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.803746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803685    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d107244-9c33-4ef4-a38d-25957142a546-ovn-node-metrics-cert\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.803746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803739    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9wgl\" (UniqueName: \"kubernetes.io/projected/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-kube-api-access-s9wgl\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803776    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-os-release\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803800    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-netns\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803820    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-cni-bin\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803835    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-lib-modules\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803850    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-systemd\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803863    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-log-socket\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803877    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-cni-multus\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803892    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-hostroot\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.803925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803922    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-system-cni-dir\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803940    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-os-release\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803964    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-kubelet\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.803992    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-cni-netd\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804012    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804034    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87c5dfeb-d7d2-41df-9415-2de9e282413b-agent-certs\") pod \"konnectivity-agent-pw577\" (UID: \"87c5dfeb-d7d2-41df-9415-2de9e282413b\") " pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804052    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/839add11-09fa-4b3c-b6d3-b3e963c9c40c-host-slash\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804068    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-etc-selinux\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804086    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78ac947a-ae79-4d31-a502-ec3efb1aafd6-host\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804107    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rmm8\" (UniqueName: \"kubernetes.io/projected/78ac947a-ae79-4d31-a502-ec3efb1aafd6-kube-api-access-4rmm8\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804128    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhkr2\" (UniqueName: \"kubernetes.io/projected/4e779cdf-580d-4a00-8bd8-69514b5c3419-kube-api-access-jhkr2\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.804163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804147    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-cnibin\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804167    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-ovn\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804181    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-socket-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804203    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-sys-fs\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804221    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6pcl\" (UniqueName: \"kubernetes.io/projected/adfb44a3-6127-4465-8466-25a3786e8307-kube-api-access-p6pcl\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804236    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78ac947a-ae79-4d31-a502-ec3efb1aafd6-serviceca\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804253    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804269    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-kubernetes\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804288    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/efd9f735-7bea-4f49-8d53-b62a27a10da3-dbus\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804308    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804321    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87c5dfeb-d7d2-41df-9415-2de9e282413b-konnectivity-ca\") pod \"konnectivity-agent-pw577\" (UID: \"87c5dfeb-d7d2-41df-9415-2de9e282413b\") " pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804344    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-cni-binary-copy\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804361    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-conf-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804386    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804418    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f8lb\" (UniqueName: \"kubernetes.io/projected/088f9643-e34a-468f-b1be-432eaa0d2346-kube-api-access-8f8lb\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804456    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.804470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804472    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/839add11-09fa-4b3c-b6d3-b3e963c9c40c-iptables-alerter-script\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804494    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-registration-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804512    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9p6w\" (UniqueName: \"kubernetes.io/projected/839add11-09fa-4b3c-b6d3-b3e963c9c40c-kube-api-access-c9p6w\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804527    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-run-ovn-kubernetes\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804541    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-env-overrides\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804558    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-cni-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804584    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e779cdf-580d-4a00-8bd8-69514b5c3419-tmp\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804601    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088f9643-e34a-468f-b1be-432eaa0d2346-tmp-dir\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804623    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804647    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804674    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-device-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804691    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-modprobe-d\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804707    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-k8s-cni-cncf-io\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804722    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-sys\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804757    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/088f9643-e34a-468f-b1be-432eaa0d2346-hosts-file\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804780    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-run-netns\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.804928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804795    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-ovnkube-script-lib\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804810    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-var-lib-kubelet\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804826    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f7q2\" (UniqueName: \"kubernetes.io/projected/e851f244-643c-4512-97cc-ca6e43bbbce6-kube-api-access-4f7q2\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804861    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804887    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-multus-certs\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804906    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v46nm\" (UniqueName: \"kubernetes.io/projected/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-kube-api-access-v46nm\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804926    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-systemd\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804948    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqbpq\" (UniqueName: \"kubernetes.io/projected/9d107244-9c33-4ef4-a38d-25957142a546-kube-api-access-nqbpq\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.804975    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-kubelet\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805001    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805019    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-systemd-units\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805046    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-node-log\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805071    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-system-cni-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805089    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-daemon-config\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805108    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-run\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805126    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/efd9f735-7bea-4f49-8d53-b62a27a10da3-kubelet-config\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.805361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805142    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-slash\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805155    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-var-lib-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805174    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-etc-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805195    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-cnibin\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805210    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805224    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysconfig\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805240    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysctl-conf\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.805816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.805260    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-socket-dir-parent\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.905670 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905623    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-os-release\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.905670 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905668    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-kubelet\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905690    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-cni-netd\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905713    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905750    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87c5dfeb-d7d2-41df-9415-2de9e282413b-agent-certs\") pod \"konnectivity-agent-pw577\" (UID: \"87c5dfeb-d7d2-41df-9415-2de9e282413b\") " pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905774    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/839add11-09fa-4b3c-b6d3-b3e963c9c40c-host-slash\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905777    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-os-release\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905776    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-cni-netd\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905807    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905801    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-etc-selinux\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905848    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78ac947a-ae79-4d31-a502-ec3efb1aafd6-host\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905850    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/839add11-09fa-4b3c-b6d3-b3e963c9c40c-host-slash\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905776    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-kubelet\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905867    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rmm8\" (UniqueName: \"kubernetes.io/projected/78ac947a-ae79-4d31-a502-ec3efb1aafd6-kube-api-access-4rmm8\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905892    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhkr2\" (UniqueName: \"kubernetes.io/projected/4e779cdf-580d-4a00-8bd8-69514b5c3419-kube-api-access-jhkr2\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905903    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78ac947a-ae79-4d31-a502-ec3efb1aafd6-host\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.905925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905924    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-etc-selinux\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.905958    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-cnibin\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906081    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-cnibin\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906094    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-ovn\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906120    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-socket-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906131    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-ovn\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906138    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-sys-fs\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906160    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6pcl\" (UniqueName: \"kubernetes.io/projected/adfb44a3-6127-4465-8466-25a3786e8307-kube-api-access-p6pcl\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906180    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78ac947a-ae79-4d31-a502-ec3efb1aafd6-serviceca\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906195    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906209    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-kubernetes\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906208    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-sys-fs\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906227    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/efd9f735-7bea-4f49-8d53-b62a27a10da3-dbus\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906296    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-kubernetes\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906311    2544 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906356    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906358    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/efd9f735-7bea-4f49-8d53-b62a27a10da3-dbus\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.906505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906381    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-socket-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906407    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87c5dfeb-d7d2-41df-9415-2de9e282413b-konnectivity-ca\") pod \"konnectivity-agent-pw577\" (UID: \"87c5dfeb-d7d2-41df-9415-2de9e282413b\") " pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906433    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-cni-binary-copy\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.906455    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906473    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-conf-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906497    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.906567    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:24.406517858 +0000 UTC m=+63.064227366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906585    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f8lb\" (UniqueName: \"kubernetes.io/projected/088f9643-e34a-468f-b1be-432eaa0d2346-kube-api-access-8f8lb\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906629    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906655    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/839add11-09fa-4b3c-b6d3-b3e963c9c40c-iptables-alerter-script\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906682    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-registration-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906703    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-conf-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906708    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78ac947a-ae79-4d31-a502-ec3efb1aafd6-serviceca\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906720    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9p6w\" (UniqueName: \"kubernetes.io/projected/839add11-09fa-4b3c-b6d3-b3e963c9c40c-kube-api-access-c9p6w\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906768    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-run-ovn-kubernetes\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906793    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-env-overrides\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906825    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-cni-binary-copy\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.907310 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906832    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-cni-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906856    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e779cdf-580d-4a00-8bd8-69514b5c3419-tmp\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906881    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088f9643-e34a-468f-b1be-432eaa0d2346-tmp-dir\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906922    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906948    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906974    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.906983    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/87c5dfeb-d7d2-41df-9415-2de9e282413b-konnectivity-ca\") pod \"konnectivity-agent-pw577\" (UID: \"87c5dfeb-d7d2-41df-9415-2de9e282413b\") " pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907008    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-device-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907035    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-modprobe-d\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907036    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-cni-binary-copy\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907048    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-cni-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907078    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-k8s-cni-cncf-io\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907102    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-sys\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907107    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907135    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/088f9643-e34a-468f-b1be-432eaa0d2346-hosts-file\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907162    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-run-netns\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907177    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-sys\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.908107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907183    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-ovnkube-script-lib\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907229    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-device-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907242    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/839add11-09fa-4b3c-b6d3-b3e963c9c40c-iptables-alerter-script\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907294    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-run-netns\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907303    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-modprobe-d\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907319    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/088f9643-e34a-468f-b1be-432eaa0d2346-tmp-dir\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907341    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/088f9643-e34a-468f-b1be-432eaa0d2346-hosts-file\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907351    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-k8s-cni-cncf-io\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907379    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-env-overrides\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907396    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/adfb44a3-6127-4465-8466-25a3786e8307-registration-dir\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.907424    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907436    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-run-ovn-kubernetes\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907466    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-var-lib-kubelet\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907490    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f7q2\" (UniqueName: \"kubernetes.io/projected/e851f244-643c-4512-97cc-ca6e43bbbce6-kube-api-access-4f7q2\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.907538    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:24.407522353 +0000 UTC m=+63.065231880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907553    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907574    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-multus-certs\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.908913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907593    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v46nm\" (UniqueName: \"kubernetes.io/projected/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-kube-api-access-v46nm\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907623    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-systemd\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907644    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqbpq\" (UniqueName: \"kubernetes.io/projected/9d107244-9c33-4ef4-a38d-25957142a546-kube-api-access-nqbpq\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907643    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907650    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907493    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-var-lib-kubelet\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907663    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-kubelet\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907688    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-multus-certs\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907691    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907726    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-kubelet\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907755    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-systemd-units\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907756    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-systemd\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907854    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-systemd-units\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907886    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-node-log\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907921    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-ovnkube-script-lib\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907929    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-system-cni-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907955    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-daemon-config\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907974    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-node-log\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.909489 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.907978    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-run\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908003    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/efd9f735-7bea-4f49-8d53-b62a27a10da3-kubelet-config\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908022    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-system-cni-dir\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908028    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-slash\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908061    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-var-lib-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908086    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-etc-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908102    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/efd9f735-7bea-4f49-8d53-b62a27a10da3-kubelet-config\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908106    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-cnibin\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908092    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-run\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908137    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908063    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-slash\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908149    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-cnibin\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908161    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e851f244-643c-4512-97cc-ca6e43bbbce6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908164    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysconfig\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908184    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-var-lib-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908189    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-etc-openvswitch\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908209    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysctl-conf\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908234    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysconfig\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910038 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908240    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-socket-dir-parent\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908267    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-etc-kubernetes\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908281    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-socket-dir-parent\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908296    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysctl-conf\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908322    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysctl-d\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908366    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-host\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908373    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-etc-kubernetes\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908386    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-tuned\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908418    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-cni-bin\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908421    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-multus-daemon-config\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908457    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-ovnkube-config\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908464    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-sysctl-d\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908418    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-host\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908481    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d107244-9c33-4ef4-a38d-25957142a546-ovn-node-metrics-cert\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908503    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-host-cni-bin\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908507    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9wgl\" (UniqueName: \"kubernetes.io/projected/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-kube-api-access-s9wgl\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908530    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-os-release\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908550    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-netns\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.910562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908569    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-cni-bin\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908590    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-lib-modules\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908630    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-systemd\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908633    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-run-netns\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908676    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-run-systemd\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908697    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-os-release\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908693    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-log-socket\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908722    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9d107244-9c33-4ef4-a38d-25957142a546-log-socket\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908765    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-cni-multus\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908787    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-cni-bin\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908815    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e779cdf-580d-4a00-8bd8-69514b5c3419-lib-modules\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908792    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-hostroot\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908853    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-host-var-lib-cni-multus\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908863    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-system-cni-dir\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908835    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-hostroot\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.908938    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e851f244-643c-4512-97cc-ca6e43bbbce6-system-cni-dir\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.909450    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9d107244-9c33-4ef4-a38d-25957142a546-ovnkube-config\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.911034    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4e779cdf-580d-4a00-8bd8-69514b5c3419-etc-tuned\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.911206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.911060    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e779cdf-580d-4a00-8bd8-69514b5c3419-tmp\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.911673 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.911113    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9d107244-9c33-4ef4-a38d-25957142a546-ovn-node-metrics-cert\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.911673 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.911187    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/87c5dfeb-d7d2-41df-9415-2de9e282413b-agent-certs\") pod \"konnectivity-agent-pw577\" (UID: \"87c5dfeb-d7d2-41df-9415-2de9e282413b\") " pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:23.914070 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.914048    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:23.914070 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.914070    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:23.914247 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.914080    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:23.914247 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:23.914131    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:24.414113434 +0000 UTC m=+63.071822948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:23.915204 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.915177    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rmm8\" (UniqueName: \"kubernetes.io/projected/78ac947a-ae79-4d31-a502-ec3efb1aafd6-kube-api-access-4rmm8\") pod \"node-ca-xg9lr\" (UID: \"78ac947a-ae79-4d31-a502-ec3efb1aafd6\") " pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:23.916062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.916040    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f8lb\" (UniqueName: \"kubernetes.io/projected/088f9643-e34a-468f-b1be-432eaa0d2346-kube-api-access-8f8lb\") pod \"node-resolver-757g8\" (UID: \"088f9643-e34a-468f-b1be-432eaa0d2346\") " pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:23.916506 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.916487    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhkr2\" (UniqueName: \"kubernetes.io/projected/4e779cdf-580d-4a00-8bd8-69514b5c3419-kube-api-access-jhkr2\") pod \"tuned-8dcff\" (UID: \"4e779cdf-580d-4a00-8bd8-69514b5c3419\") " pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:23.916629 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.916612    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9p6w\" (UniqueName: \"kubernetes.io/projected/839add11-09fa-4b3c-b6d3-b3e963c9c40c-kube-api-access-c9p6w\") pod \"iptables-alerter-s6fhr\" (UID: \"839add11-09fa-4b3c-b6d3-b3e963c9c40c\") " pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:23.917104 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.917080    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9wgl\" (UniqueName: \"kubernetes.io/projected/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-kube-api-access-s9wgl\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:23.917472 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.917456    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f7q2\" (UniqueName: \"kubernetes.io/projected/e851f244-643c-4512-97cc-ca6e43bbbce6-kube-api-access-4f7q2\") pod \"multus-additional-cni-plugins-xk5wh\" (UID: \"e851f244-643c-4512-97cc-ca6e43bbbce6\") " pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:23.917625 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.917604    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqbpq\" (UniqueName: \"kubernetes.io/projected/9d107244-9c33-4ef4-a38d-25957142a546-kube-api-access-nqbpq\") pod \"ovnkube-node-mvvlt\" (UID: \"9d107244-9c33-4ef4-a38d-25957142a546\") " pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:23.918485 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.918464    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6pcl\" (UniqueName: \"kubernetes.io/projected/adfb44a3-6127-4465-8466-25a3786e8307-kube-api-access-p6pcl\") pod \"aws-ebs-csi-driver-node-lsrp4\" (UID: \"adfb44a3-6127-4465-8466-25a3786e8307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:23.918624 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.918610    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v46nm\" (UniqueName: \"kubernetes.io/projected/5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c-kube-api-access-v46nm\") pod \"multus-qqs98\" (UID: \"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c\") " pod="openshift-multus/multus-qqs98"
Jul 27 19:27:23.937566 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:23.937522    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal" podStartSLOduration=0.937508632 podStartE2EDuration="937.508632ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:23.93723625 +0000 UTC m=+62.594945781" watchObservedRunningTime="2026-07-27 19:27:23.937508632 +0000 UTC m=+62.595218161"
Jul 27 19:27:24.015666 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.015562    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xk5wh"
Jul 27 19:27:24.021608 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.021584    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qqs98"
Jul 27 19:27:24.027382 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.027354    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s6fhr"
Jul 27 19:27:24.028342 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.028309    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode851f244_643c_4512_97cc_ca6e43bbbce6.slice/crio-9d9f19446551c3b988c4196d9441ca812c1c13105295c2874685ee5135af9331 WatchSource:0}: Error finding container 9d9f19446551c3b988c4196d9441ca812c1c13105295c2874685ee5135af9331: Status 404 returned error can't find the container with id 9d9f19446551c3b988c4196d9441ca812c1c13105295c2874685ee5135af9331
Jul 27 19:27:24.029960 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.029935    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb7d020_8ad1_4b43_b7a5_1099d16ebd9c.slice/crio-3db7de8bba5ea572608f2ba8d6a3368731a7cdb17e07bd3483ae5e30da12d95c WatchSource:0}: Error finding container 3db7de8bba5ea572608f2ba8d6a3368731a7cdb17e07bd3483ae5e30da12d95c: Status 404 returned error can't find the container with id 3db7de8bba5ea572608f2ba8d6a3368731a7cdb17e07bd3483ae5e30da12d95c
Jul 27 19:27:24.033577 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.033551    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:24.036002 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.035969    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839add11_09fa_4b3c_b6d3_b3e963c9c40c.slice/crio-58ab45bd104ae488dd7cea00244039cb381d86cd795a8e8658dc6a6e89c798e0 WatchSource:0}: Error finding container 58ab45bd104ae488dd7cea00244039cb381d86cd795a8e8658dc6a6e89c798e0: Status 404 returned error can't find the container with id 58ab45bd104ae488dd7cea00244039cb381d86cd795a8e8658dc6a6e89c798e0
Jul 27 19:27:24.038365 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.038346    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xg9lr"
Jul 27 19:27:24.041981 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.041962    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d107244_9c33_4ef4_a38d_25957142a546.slice/crio-4616acd0ef6aa048385d27cfc82ade95523753d36b34e5340f1842790ee5d0f9 WatchSource:0}: Error finding container 4616acd0ef6aa048385d27cfc82ade95523753d36b34e5340f1842790ee5d0f9: Status 404 returned error can't find the container with id 4616acd0ef6aa048385d27cfc82ade95523753d36b34e5340f1842790ee5d0f9
Jul 27 19:27:24.042826 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.042810    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4"
Jul 27 19:27:24.046978 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.046954    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ac947a_ae79_4d31_a502_ec3efb1aafd6.slice/crio-053ee4d7c09fa14964601d0c285d1bd5b25f0a80c7ebd486ee74db1b0cd7216b WatchSource:0}: Error finding container 053ee4d7c09fa14964601d0c285d1bd5b25f0a80c7ebd486ee74db1b0cd7216b: Status 404 returned error can't find the container with id 053ee4d7c09fa14964601d0c285d1bd5b25f0a80c7ebd486ee74db1b0cd7216b
Jul 27 19:27:24.047595 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.047576    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:24.052410 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.052381    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dcff"
Jul 27 19:27:24.053486 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.053459    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadfb44a3_6127_4465_8466_25a3786e8307.slice/crio-02f642e585ae9a8e9abd92a86fef865a23de54dfef3003bfb2d7365017af8d5e WatchSource:0}: Error finding container 02f642e585ae9a8e9abd92a86fef865a23de54dfef3003bfb2d7365017af8d5e: Status 404 returned error can't find the container with id 02f642e585ae9a8e9abd92a86fef865a23de54dfef3003bfb2d7365017af8d5e
Jul 27 19:27:24.055351 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.055335    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-757g8"
Jul 27 19:27:24.058638 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.058616    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c5dfeb_d7d2_41df_9415_2de9e282413b.slice/crio-8e35c56522c1f0927b77f37cbda1b147c526a71b1d35b9fe1452a64091ec8203 WatchSource:0}: Error finding container 8e35c56522c1f0927b77f37cbda1b147c526a71b1d35b9fe1452a64091ec8203: Status 404 returned error can't find the container with id 8e35c56522c1f0927b77f37cbda1b147c526a71b1d35b9fe1452a64091ec8203
Jul 27 19:27:24.062075 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.061931    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e779cdf_580d_4a00_8bd8_69514b5c3419.slice/crio-99bf727f5c99bd7486606f506817e96f86c42b4f18a0c9a9c8484655835c0972 WatchSource:0}: Error finding container 99bf727f5c99bd7486606f506817e96f86c42b4f18a0c9a9c8484655835c0972: Status 404 returned error can't find the container with id 99bf727f5c99bd7486606f506817e96f86c42b4f18a0c9a9c8484655835c0972
Jul 27 19:27:24.064914 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:27:24.064885    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088f9643_e34a_468f_b1be_432eaa0d2346.slice/crio-c026c8180cbb59341a01b07ffa7110ac17f752cc0f1c5d1ac5301b478da6e171 WatchSource:0}: Error finding container c026c8180cbb59341a01b07ffa7110ac17f752cc0f1c5d1ac5301b478da6e171: Status 404 returned error can't find the container with id c026c8180cbb59341a01b07ffa7110ac17f752cc0f1c5d1ac5301b478da6e171
Jul 27 19:27:24.412661 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.412625    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:24.412661 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.412665    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:24.412990 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.412787    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:24.412990 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.412810    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:24.412990 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.412844    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:25.412829543 +0000 UTC m=+64.070539051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:24.412990 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.412892    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:25.412885058 +0000 UTC m=+64.070594565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:24.513975 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.513933    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:24.514178 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.514159    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:24.514244 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.514190    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:24.514244 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.514203    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:24.514359 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.514261    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:25.514243371 +0000 UTC m=+64.171952891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:24.669867 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.669789    2544 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:24.788994 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.788954    2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:22:23 +0000 UTC" deadline="2028-01-10 00:09:44.585497484 +0000 UTC"
Jul 27 19:27:24.788994 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.788992    2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12748h42m19.796510439s"
Jul 27 19:27:24.835327 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.834811    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:24.835327 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:24.834947    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:24.963781 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.963687    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-757g8" event={"ID":"088f9643-e34a-468f-b1be-432eaa0d2346","Type":"ContainerStarted","Data":"c026c8180cbb59341a01b07ffa7110ac17f752cc0f1c5d1ac5301b478da6e171"}
Jul 27 19:27:24.974261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.974225    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dcff" event={"ID":"4e779cdf-580d-4a00-8bd8-69514b5c3419","Type":"ContainerStarted","Data":"99bf727f5c99bd7486606f506817e96f86c42b4f18a0c9a9c8484655835c0972"}
Jul 27 19:27:24.980639 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.980605    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xg9lr" event={"ID":"78ac947a-ae79-4d31-a502-ec3efb1aafd6","Type":"ContainerStarted","Data":"053ee4d7c09fa14964601d0c285d1bd5b25f0a80c7ebd486ee74db1b0cd7216b"}
Jul 27 19:27:24.994748 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:24.994701    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"4616acd0ef6aa048385d27cfc82ade95523753d36b34e5340f1842790ee5d0f9"}
Jul 27 19:27:25.011173 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.011140    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s6fhr" event={"ID":"839add11-09fa-4b3c-b6d3-b3e963c9c40c","Type":"ContainerStarted","Data":"58ab45bd104ae488dd7cea00244039cb381d86cd795a8e8658dc6a6e89c798e0"}
Jul 27 19:27:25.020487 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.020451    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerStarted","Data":"9d9f19446551c3b988c4196d9441ca812c1c13105295c2874685ee5135af9331"}
Jul 27 19:27:25.024917 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.024883    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pw577" event={"ID":"87c5dfeb-d7d2-41df-9415-2de9e282413b","Type":"ContainerStarted","Data":"8e35c56522c1f0927b77f37cbda1b147c526a71b1d35b9fe1452a64091ec8203"}
Jul 27 19:27:25.031693 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.031629    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4" event={"ID":"adfb44a3-6127-4465-8466-25a3786e8307","Type":"ContainerStarted","Data":"02f642e585ae9a8e9abd92a86fef865a23de54dfef3003bfb2d7365017af8d5e"}
Jul 27 19:27:25.040914 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.040856    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qqs98" event={"ID":"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c","Type":"ContainerStarted","Data":"3db7de8bba5ea572608f2ba8d6a3368731a7cdb17e07bd3483ae5e30da12d95c"}
Jul 27 19:27:25.421075 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.421009    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:25.421075 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.421063    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:25.421622 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.421220    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:25.421622 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.421283    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:27.421264831 +0000 UTC m=+66.078974343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:25.421774 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.421676    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:25.421774 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.421725    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:27.4217103 +0000 UTC m=+66.079419811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:25.521672 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.521624    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:25.521925 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.521907    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:25.522002 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.521934    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:25.522002 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.521946    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:25.522110 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.522006    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:27.521988304 +0000 UTC m=+66.179697824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:25.790269 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.790174    2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 19:22:23 +0000 UTC" deadline="2028-02-19 22:46:57.740444498 +0000 UTC"
Jul 27 19:27:25.790269 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.790212    2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13731h19m31.950237376s"
Jul 27 19:27:25.836363 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.836331    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:25.836564 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.836492    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:25.836692 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:25.836649    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:25.836822 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:25.836770    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:26.835190 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:26.835148    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:26.835657 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:26.835282    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:27.438540 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:27.438468    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:27.438540 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:27.438517    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:27.438906 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.438670    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:27.438906 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.438698    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:27.438906 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.438747    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:31.43871332 +0000 UTC m=+70.096422830 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:27.438906 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.438783    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:31.438765479 +0000 UTC m=+70.096474993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:27.539235 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:27.539137    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:27.539434 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.539334    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:27.539434 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.539359    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:27.539434 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.539373    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:27.539434 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.539437    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:31.539417267 +0000 UTC m=+70.197126810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:27.839833 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:27.839596    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:27.840270 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:27.840201    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:27.840329 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.840310    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:27.840473 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:27.839714    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:28.835159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:28.835088    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:28.835349 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:28.835224    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:29.835051 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:29.835010    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:29.835520 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:29.835145    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:29.835520 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:29.835195    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:29.835520 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:29.835307    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:30.834242 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:30.834207    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:30.834424 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:30.834362    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:31.471823 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:31.471781    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:31.472260 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:31.471833    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:31.472260 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.471956    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:31.472260 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.472036    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:39.472015295 +0000 UTC m=+78.129724826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:31.472462 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.471956    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:31.472462 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.472459    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:39.472440653 +0000 UTC m=+78.130150208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:31.572537 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:31.572497    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:31.572715 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.572663    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:31.572715 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.572681    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:31.572715 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.572691    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:31.572923 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.572756    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:39.572724133 +0000 UTC m=+78.230433644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:31.834659 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:31.834628    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:31.834831 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.834767    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:31.836208 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:31.836184    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:31.836319 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:31.836280    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:32.834806 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:32.834767    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:32.835215 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:32.834899    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:33.838012 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:33.837982    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:33.838473 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:33.838112    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:33.838473 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:33.837982    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:33.838473 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:33.838408    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:34.834772 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:34.834720    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:34.834949 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:34.834858    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:35.834622 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:35.834581    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:35.835095 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:35.834716    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:35.835095 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:35.834763    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:35.835095 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:35.834873    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:36.834560 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:36.834525    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:36.834819 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:36.834636    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:37.835155 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:37.835117    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:37.835614 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:37.835253    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:37.835614 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:37.835296    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:37.835614 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:37.835402    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:38.834783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:38.834747    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:38.834963 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:38.834876    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:39.530009 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:39.529970    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:39.530009 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:39.530012    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:39.530472 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.530154    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:39.530472 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.530202    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:39.530472 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.530236    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:55.53021593 +0000 UTC m=+94.187925453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:39.530472 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.530261    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:55.530244548 +0000 UTC m=+94.187954077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:39.630643 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:39.630588    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:39.630855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.630795    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:39.630855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.630822    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:39.630855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.630837    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:39.631001 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.630898    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:27:55.630880169 +0000 UTC m=+94.288589676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:39.837889 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:39.837816    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:39.838044 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:39.837816    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:39.838044 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.837935    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:39.838122 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:39.838043    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:40.834450 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:40.834416    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:40.835024 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:40.834531    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:41.836236 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:41.834884    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:41.836236 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:41.835012    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:41.837483 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:41.837457    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:41.837592 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:41.837561    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:42.078240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.077931    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pw577" event={"ID":"87c5dfeb-d7d2-41df-9415-2de9e282413b","Type":"ContainerStarted","Data":"fb3b498ef279c32108440c496bfbcdb092b373db85fd1970235527a57a1d7f3c"}
Jul 27 19:27:42.079590 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.079561    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4" event={"ID":"adfb44a3-6127-4465-8466-25a3786e8307","Type":"ContainerStarted","Data":"cfca2ec5f9e12bec822a07611bf5b8831c59ec64fb524d76430f399c7f438de6"}
Jul 27 19:27:42.081283 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.081260    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qqs98" event={"ID":"5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c","Type":"ContainerStarted","Data":"f21a4505335f30b190114f32f77f792241e5691f26b5d329d5715ba2c95dead7"}
Jul 27 19:27:42.083394 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.083371    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_91a72cf2fcdcd15b4bb9ab9ab54878da/setup/2.log"
Jul 27 19:27:42.083486 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.083414    2544 generic.go:356] "Generic (PLEG): container finished" podID="91a72cf2fcdcd15b4bb9ab9ab54878da" containerID="79d6934e7beb4ab028c9e8cf2401e84e63a83e655e82e1b83f038e58593a0005" exitCode=0
Jul 27 19:27:42.083545 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.083485    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerDied","Data":"79d6934e7beb4ab028c9e8cf2401e84e63a83e655e82e1b83f038e58593a0005"}
Jul 27 19:27:42.083545 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.083532    2544 scope.go:117] "RemoveContainer" containerID="b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b"
Jul 27 19:27:42.083861 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.083802    2544 scope.go:117] "RemoveContainer" containerID="b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b"
Jul 27 19:27:42.085634 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.085335    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-757g8" event={"ID":"088f9643-e34a-468f-b1be-432eaa0d2346","Type":"ContainerStarted","Data":"a79c8376b03a12db9a81203e8df4a8da58aa4d18d44438d8921fb2046be10189"}
Jul 27 19:27:42.087210 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.087062    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dcff" event={"ID":"4e779cdf-580d-4a00-8bd8-69514b5c3419","Type":"ContainerStarted","Data":"294a7f74e0e5f19443203484a671ff4e7bcc0ca5299ad16876a409f841c94da6"}
Jul 27 19:27:42.088836 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.088784    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xg9lr" event={"ID":"78ac947a-ae79-4d31-a502-ec3efb1aafd6","Type":"ContainerStarted","Data":"ba4e3398ec1bc1193d7a4b7364a150eaed36f744afbbe6d790ace107d974efd2"}
Jul 27 19:27:42.094385 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.094274    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"3debae0a6d7aba5ad51402ee255b6f63ee6ea43daae0d8ff7aaca8a70d7acbb3"}
Jul 27 19:27:42.094862 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:42.094826    2544 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator_91a72cf2fcdcd15b4bb9ab9ab54878da_2 in pod sandbox 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 from index: no such id: 'b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b'" containerID="b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b"
Jul 27 19:27:42.094971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.094869    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b"} err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal_openshift-machine-config-operator_91a72cf2fcdcd15b4bb9ab9ab54878da_2 in pod sandbox 581b3348b64c4c61f103de5427099fc6a6e86cc1056a9bdf1746ac74c3018657 from index: no such id: 'b8ac96eb3e313747797b5c5f22ddc405b7b651eaf249bf15e1b9afaeb6cf0c9b'"
Jul 27 19:27:42.097407 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.097383    2544 generic.go:356] "Generic (PLEG): container finished" podID="e851f244-643c-4512-97cc-ca6e43bbbce6" containerID="b8d822c644f5df5c246e1e6c127b341faefe6fe9d6f0937c3d54e893b35fd982" exitCode=0
Jul 27 19:27:42.097516 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.097418    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerDied","Data":"b8d822c644f5df5c246e1e6c127b341faefe6fe9d6f0937c3d54e893b35fd982"}
Jul 27 19:27:42.098357 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.098026    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pw577" podStartSLOduration=16.196403357 podStartE2EDuration="26.098012236s" totalImagesPullingTime="9.901608879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:42.097589734 +0000 UTC m=+80.755299265" watchObservedRunningTime="2026-07-27 19:27:42.098012236 +0000 UTC m=+80.755721764"
Jul 27 19:27:42.116399 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.116346    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xg9lr" podStartSLOduration=8.48035949 podStartE2EDuration="26.11633129s" totalImagesPullingTime="17.6359718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:42.11562282 +0000 UTC m=+80.773332349" watchObservedRunningTime="2026-07-27 19:27:42.11633129 +0000 UTC m=+80.774040856"
Jul 27 19:27:42.172809 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.172759    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qqs98" podStartSLOduration=8.484057166 podStartE2EDuration="26.172740107s" totalImagesPullingTime="17.688682941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:42.172653746 +0000 UTC m=+80.830363278" watchObservedRunningTime="2026-07-27 19:27:42.172740107 +0000 UTC m=+80.830449629"
Jul 27 19:27:42.189698 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.189656    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8dcff" podStartSLOduration=8.565932216 podStartE2EDuration="26.189630428s" totalImagesPullingTime="17.623698212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:42.189351444 +0000 UTC m=+80.847060975" watchObservedRunningTime="2026-07-27 19:27:42.189630428 +0000 UTC m=+80.847339955"
Jul 27 19:27:42.206031 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.205974    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-757g8" podStartSLOduration=8.588065871 podStartE2EDuration="26.205950443s" totalImagesPullingTime="17.617884572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:42.204865928 +0000 UTC m=+80.862575459" watchObservedRunningTime="2026-07-27 19:27:42.205950443 +0000 UTC m=+80.863659973"
Jul 27 19:27:42.757987 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.757819    2544 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 19:27:42.792725 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.792610    2544 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T19:27:42.757982963Z","UUID":"992bd881-78a3-4c5f-93d8-7adb1ca86f6a","Handler":null,"Name":"","Endpoint":""}
Jul 27 19:27:42.794302 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.794276    2544 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 19:27:42.794435 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.794324    2544 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 19:27:42.834972 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:42.834945    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:42.835130 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:42.835071    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:43.102746 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.102697    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"91a72cf2fcdcd15b4bb9ab9ab54878da","Type":"ContainerStarted","Data":"55abbd28e68c88bf927c23a4c1349a450af87749c0d4c2ca7c7a3b0336ce24dd"}
Jul 27 19:27:43.106222 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.106192    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"936b6d063592dc5c7ed4f76babc882a44046a0d058e435450ca6478e0339f51b"}
Jul 27 19:27:43.106360 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.106276    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"52ba184314d5fcc185d61255b61b373bdbb421023b678d4080f0ea5e7c7059a0"}
Jul 27 19:27:43.106360 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.106325    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"9b1f4d48104a345515bb5135da9e54ad4e466fbd889eb785a4dad68e206eab5c"}
Jul 27 19:27:43.106477 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.106368    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"a0c02cd153faa146bfc1befef56a2d96195caadf35de376fb213386f525d6492"}
Jul 27 19:27:43.106477 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.106412    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"0b9e5357135c7d1f3719cca90083f9667de29a9e6bddc6fad5ad38d6dd0343c5"}
Jul 27 19:27:43.108510 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.108480    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4" event={"ID":"adfb44a3-6127-4465-8466-25a3786e8307","Type":"ContainerStarted","Data":"d9bfad81409dad479803663eb76e5db44a40e34b897aa7ad11f6fee2a8ffe13e"}
Jul 27 19:27:43.118952 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.118875    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" podStartSLOduration=20.118862097 podStartE2EDuration="20.118862097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:43.118641781 +0000 UTC m=+81.776351309" watchObservedRunningTime="2026-07-27 19:27:43.118862097 +0000 UTC m=+81.776571625"
Jul 27 19:27:43.443833 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.443805    2544 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:43.835285 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.835248    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:43.835436 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:43.835252    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:43.835436 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:43.835389    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:43.835529 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:43.835468    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:44.048479 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.048377    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:44.049579 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.049554    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:44.112318 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.112279    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s6fhr" event={"ID":"839add11-09fa-4b3c-b6d3-b3e963c9c40c","Type":"ContainerStarted","Data":"7f2a7aac0407f6a9f0fda8ff5ff1eb11679fe1c24a7ab962441e39b433e302b5"}
Jul 27 19:27:44.114748 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.114700    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4" event={"ID":"adfb44a3-6127-4465-8466-25a3786e8307","Type":"ContainerStarted","Data":"d5b5a3e13b62ff51848571058df68064bb9a5bdeed6ba17692a6213d6fe0bfbb"}
Jul 27 19:27:44.115019 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.114997    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:44.115627 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.115610    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pw577"
Jul 27 19:27:44.131085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.131037    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s6fhr" podStartSLOduration=10.48562309 podStartE2EDuration="28.131021523s" totalImagesPullingTime="17.645398433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:44.130442777 +0000 UTC m=+82.788152306" watchObservedRunningTime="2026-07-27 19:27:44.131021523 +0000 UTC m=+82.788731051"
Jul 27 19:27:44.150131 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.150070    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lsrp4" podStartSLOduration=8.843138669 podStartE2EDuration="28.150055196s" totalImagesPullingTime="19.306916527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:44.149726795 +0000 UTC m=+82.807436325" watchObservedRunningTime="2026-07-27 19:27:44.150055196 +0000 UTC m=+82.807764720"
Jul 27 19:27:44.834748 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:44.834706    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:44.834916 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:44.834873    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:45.120724 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:45.120687    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"f82cf4fb5beba74196e23c4aa8d498f8434eb271f63ca43a4b9a423f3f411d38"}
Jul 27 19:27:45.834680 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:45.834646    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:45.834888 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:45.834784    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:45.834888 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:45.834836    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:45.834999 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:45.834958    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:46.835418 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:46.835152    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:46.835940 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:46.835440    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:47.129636 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.129594    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" event={"ID":"9d107244-9c33-4ef4-a38d-25957142a546","Type":"ContainerStarted","Data":"bfc28c6ed16ba1e3840561a34b02d09d582f1d5c6b768466e9172d761b514baa"}
Jul 27 19:27:47.129932 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.129910    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:47.130036 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.129986    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:47.130036 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.130033    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:47.131959 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.131935    2544 generic.go:356] "Generic (PLEG): container finished" podID="e851f244-643c-4512-97cc-ca6e43bbbce6" containerID="14eb189f92a7d47bdce943adf21780a421cc585814d139e3b025b6aad4d2ee62" exitCode=0
Jul 27 19:27:47.132082 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.131967    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerDied","Data":"14eb189f92a7d47bdce943adf21780a421cc585814d139e3b025b6aad4d2ee62"}
Jul 27 19:27:47.145842 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.145819    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:47.145958 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.145914    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:27:47.160203 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.160160    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt" podStartSLOduration=13.446353807 podStartE2EDuration="31.160147874s" totalImagesPullingTime="17.713794067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:47.159889948 +0000 UTC m=+85.817599476" watchObservedRunningTime="2026-07-27 19:27:47.160147874 +0000 UTC m=+85.817857403"
Jul 27 19:27:47.834636 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.834607    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:47.834825 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:47.834571    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:47.834825 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:47.834773    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:47.835119 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:47.835098    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:48.518819 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:48.518575    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jks2p"]
Jul 27 19:27:48.519204 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:48.518907    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:48.519204 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:48.519003    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:48.524088 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:48.522275    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2lglj"]
Jul 27 19:27:48.524088 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:48.522439    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:48.524088 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:48.522563    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:48.526031 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:48.526001    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7d4b9"]
Jul 27 19:27:48.526157 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:48.526124    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:48.526247 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:48.526229    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:49.138100 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:49.138069    2544 generic.go:356] "Generic (PLEG): container finished" podID="e851f244-643c-4512-97cc-ca6e43bbbce6" containerID="7a7edaee93eabadbc3b21c75790c0abff32355666c1b579ddf5a6fb317d1cbad" exitCode=0
Jul 27 19:27:49.138315 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:49.138157    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerDied","Data":"7a7edaee93eabadbc3b21c75790c0abff32355666c1b579ddf5a6fb317d1cbad"}
Jul 27 19:27:49.834530 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:49.834487    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:49.834962 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:49.834634    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:50.834866 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:50.834783    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:50.834866 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:50.834819    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:50.835337 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:50.834894    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:50.835337 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:50.835000    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:51.145501 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:51.145470    2544 generic.go:356] "Generic (PLEG): container finished" podID="e851f244-643c-4512-97cc-ca6e43bbbce6" containerID="056230327c92b12c2387e285f494c15a35fa969253daefc4b98b236c742f0da6" exitCode=0
Jul 27 19:27:51.145760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:51.145526    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerDied","Data":"056230327c92b12c2387e285f494c15a35fa969253daefc4b98b236c742f0da6"}
Jul 27 19:27:51.835668 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:51.835634    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:51.836458 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:51.835776    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:52.834783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:52.834668    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:52.834783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:52.834702    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:52.835019 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:52.834840    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:52.835019 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:52.834959    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:52.939479 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:52.939443    2544 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 19:27:53.834905 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:53.834871    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:53.835063 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:53.835012    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jks2p" podUID="efd9f735-7bea-4f49-8d53-b62a27a10da3"
Jul 27 19:27:54.834431 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:54.834396    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:54.834948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:54.834396    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:54.834948 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:54.834512    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2lglj" podUID="e0698697-51b0-48d1-b634-033a77df6ad4"
Jul 27 19:27:54.834948 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:54.834631    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:27:55.550327 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.550296    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:55.550415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.550334    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:55.550460 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.550419    2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:55.550460 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.550422    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:55.550514 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.550472    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret podName:efd9f735-7bea-4f49-8d53-b62a27a10da3 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:27.550455794 +0000 UTC m=+126.208165306 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret") pod "global-pull-secret-syncer-jks2p" (UID: "efd9f735-7bea-4f49-8d53-b62a27a10da3") : object "kube-system"/"original-pull-secret" not registered
Jul 27 19:27:55.550514 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.550486    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:27.550479131 +0000 UTC m=+126.208188638 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 19:27:55.643781 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.643757    2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeReady"
Jul 27 19:27:55.643921 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.643911    2544 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 19:27:55.651227 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.651207    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:55.651345 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.651333    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 19:27:55.651379 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.651351    2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 19:27:55.651379 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.651359    2544 projected.go:196] Error preparing data for projected volume kube-api-access-zcz7c for pod openshift-network-diagnostics/network-check-target-2lglj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:55.651435 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:27:55.651404    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c podName:e0698697-51b0-48d1-b634-033a77df6ad4 nodeName:}" failed. No retries permitted until 2026-07-27 19:28:27.651391145 +0000 UTC m=+126.309100652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zcz7c" (UniqueName: "kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c") pod "network-check-target-2lglj" (UID: "e0698697-51b0-48d1-b634-033a77df6ad4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 19:27:55.834966 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.834889    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:27:55.838016 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:55.837993    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:27:56.158415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.158381    2544 generic.go:356] "Generic (PLEG): container finished" podID="e851f244-643c-4512-97cc-ca6e43bbbce6" containerID="9f6cf4dedb87b4364ec5273e0fe6c1335f6bc45d86ad959b596a4c651c58a966" exitCode=0
Jul 27 19:27:56.158571 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.158439    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerDied","Data":"9f6cf4dedb87b4364ec5273e0fe6c1335f6bc45d86ad959b596a4c651c58a966"}
Jul 27 19:27:56.834293 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.834259    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:27:56.834440 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.834259    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:27:56.837258 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.837238    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lj4rj\""
Jul 27 19:27:56.837258 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.837253    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:27:56.837638 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.837268    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:27:56.837638 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.837302    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wght4\""
Jul 27 19:27:56.837638 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:56.837268    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:27:57.163873 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:57.163673    2544 generic.go:356] "Generic (PLEG): container finished" podID="e851f244-643c-4512-97cc-ca6e43bbbce6" containerID="82dd977cabc65d52a0a2a069d73367268d09723d4e34cdd53a01d7f3aaf072ad" exitCode=0
Jul 27 19:27:57.163873 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:57.163763    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerDied","Data":"82dd977cabc65d52a0a2a069d73367268d09723d4e34cdd53a01d7f3aaf072ad"}
Jul 27 19:27:58.169517 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:58.169477    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" event={"ID":"e851f244-643c-4512-97cc-ca6e43bbbce6","Type":"ContainerStarted","Data":"e12072ec787f47a18e941ea412571fe9f3acff70ac948bb85810bfadcf0115ee"}
Jul 27 19:27:58.195296 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:27:58.195247    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xk5wh" podStartSLOduration=13.607919925000001 podStartE2EDuration="42.195230293s" totalImagesPullingTime="28.587310368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:27:58.1918511 +0000 UTC m=+96.849560631" watchObservedRunningTime="2026-07-27 19:27:58.195230293 +0000 UTC m=+96.852939823"
Jul 27 19:28:19.155348 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:19.155314    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mvvlt"
Jul 27 19:28:27.562002 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.561949    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:28:27.562002 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.562008    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:28:27.565435 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.565411    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 19:28:27.565568 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.565445    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 19:28:27.572219 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:28:27.572201    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 19:28:27.572316 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:28:27.572255    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.572240384 +0000 UTC m=+190.229949894 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : secret "metrics-daemon-secret" not found
Jul 27 19:28:27.576005 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.575989    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/efd9f735-7bea-4f49-8d53-b62a27a10da3-original-pull-secret\") pod \"global-pull-secret-syncer-jks2p\" (UID: \"efd9f735-7bea-4f49-8d53-b62a27a10da3\") " pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:28:27.644302 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.644270    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jks2p"
Jul 27 19:28:27.663331 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.663306    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:28:27.666450 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.666422    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 19:28:27.676139 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.676117    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 19:28:27.686642 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.686618    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcz7c\" (UniqueName: \"kubernetes.io/projected/e0698697-51b0-48d1-b634-033a77df6ad4-kube-api-access-zcz7c\") pod \"network-check-target-2lglj\" (UID: \"e0698697-51b0-48d1-b634-033a77df6ad4\") " pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:28:27.746806 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.746781    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wght4\""
Jul 27 19:28:27.754301 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.754280    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:28:27.790315 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.790280    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jks2p"]
Jul 27 19:28:27.875065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:27.875036    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2lglj"]
Jul 27 19:28:27.878531 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:28:27.878501    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0698697_51b0_48d1_b634_033a77df6ad4.slice/crio-e0e172ebcca0b71274e82b360de4bd1a85d4e4d78d093967a2a4cd93ace1dcfa WatchSource:0}: Error finding container e0e172ebcca0b71274e82b360de4bd1a85d4e4d78d093967a2a4cd93ace1dcfa: Status 404 returned error can't find the container with id e0e172ebcca0b71274e82b360de4bd1a85d4e4d78d093967a2a4cd93ace1dcfa
Jul 27 19:28:28.236522 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:28.236486    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2lglj" event={"ID":"e0698697-51b0-48d1-b634-033a77df6ad4","Type":"ContainerStarted","Data":"e0e172ebcca0b71274e82b360de4bd1a85d4e4d78d093967a2a4cd93ace1dcfa"}
Jul 27 19:28:28.237835 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:28.237809    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jks2p" event={"ID":"efd9f735-7bea-4f49-8d53-b62a27a10da3","Type":"ContainerStarted","Data":"6f0441e8463608d5b3d4c494de3f304f62804de85e463d940a1b5d87650b465a"}
Jul 27 19:28:32.249638 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:32.249605    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2lglj" event={"ID":"e0698697-51b0-48d1-b634-033a77df6ad4","Type":"ContainerStarted","Data":"c6d4bc3665ad5cf820dc8ac0c504416b6aa4623925f67d566988c75867370ce6"}
Jul 27 19:28:32.250069 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:32.249722    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:28:32.251062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:32.251041    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jks2p" event={"ID":"efd9f735-7bea-4f49-8d53-b62a27a10da3","Type":"ContainerStarted","Data":"d9885895b23f5a3de5acad661dc6cad3cb286a6f3c320d191a08593bafb5b2fd"}
Jul 27 19:28:32.266710 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:32.266654    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2lglj" podStartSLOduration=71.985129477 podStartE2EDuration="1m16.26663509s" totalImagesPullingTime="4.281505613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:32.2659958 +0000 UTC m=+130.923705330" watchObservedRunningTime="2026-07-27 19:28:32.26663509 +0000 UTC m=+130.924344619"
Jul 27 19:28:32.280784 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:28:32.280719    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jks2p" podStartSLOduration=71.910722339 podStartE2EDuration="1m16.280707322s" totalImagesPullingTime="4.369984983s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:28:32.280193689 +0000 UTC m=+130.937903214" watchObservedRunningTime="2026-07-27 19:28:32.280707322 +0000 UTC m=+130.938416853"
Jul 27 19:29:03.256141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:03.256112    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2lglj"
Jul 27 19:29:23.818116 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.817986    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-dwtww"]
Jul 27 19:29:23.819848 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.819833    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"]
Jul 27 19:29:23.819976 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.819958    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:23.822424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.822404    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"]
Jul 27 19:29:23.822866 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.822843    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:23.823205 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.822981    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 27 19:29:23.823205 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.822984    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.823205 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.823141    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 27 19:29:23.823394 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.823259    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.824130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.824107    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gnlhc\""
Jul 27 19:29:23.824221 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.824170    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75ccb94d97-g2mbc"]
Jul 27 19:29:23.824568 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.824551    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:23.825815 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.825793    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gb4tb\""
Jul 27 19:29:23.827002 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.826979    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 27 19:29:23.827213 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.827197    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.827913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.827896    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.828359 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.828341    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.828914 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.828725    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-c77ds\""
Jul 27 19:29:23.828914 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.828790    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 27 19:29:23.829804 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.829787    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"]
Jul 27 19:29:23.829970 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.829953    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.830959 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.830842    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 27 19:29:23.830959 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.830847    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 27 19:29:23.831621 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.831602    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"]
Jul 27 19:29:23.831709 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.831652    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.831901 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.831885    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:23.833168 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.833150    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 27 19:29:23.833586 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.833570    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 27 19:29:23.833926 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.833907    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-62tqr"]
Jul 27 19:29:23.834027 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834006    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:23.834157 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834106    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-txhl6\""
Jul 27 19:29:23.834157 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834122    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 27 19:29:23.834157 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834146    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 27 19:29:23.834316 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834209    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 27 19:29:23.834534 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834514    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 27 19:29:23.834685 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.834638    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vbcg6\""
Jul 27 19:29:23.836156 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.836138    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.836752 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.836720    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 27 19:29:23.836862 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.836772    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.836920 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.836901    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-c4n7x\""
Jul 27 19:29:23.836993 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.836973    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.838551 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.838532    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.838824 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.838806    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 27 19:29:23.838994 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.838944    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.839095 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.838878    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 27 19:29:23.839159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.838825    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tgh96\""
Jul 27 19:29:23.840192 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.840176    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"]
Jul 27 19:29:23.841390 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.841373    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 27 19:29:23.842486 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.842469    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"]
Jul 27 19:29:23.842784 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.842765    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:23.845580 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.845564    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-645bb56f9f-b8xdb"]
Jul 27 19:29:23.845690 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.845676    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"
Jul 27 19:29:23.845690 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.845682    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 27 19:29:23.845817 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.845716    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.845913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.845896    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 27 19:29:23.846067 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.846047    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.846256 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.845566    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 27 19:29:23.846350 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.846132    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hlm52\""
Jul 27 19:29:23.850224 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.850200    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.850482 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.850462    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.850558 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.850507    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4f6m2\""
Jul 27 19:29:23.850832 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.850814    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zqrl2"]
Jul 27 19:29:23.852173 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.852154    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:23.853238 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.853221    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"]
Jul 27 19:29:23.853417 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.853388    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-dwtww"]
Jul 27 19:29:23.853512 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.853425    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"]
Jul 27 19:29:23.853582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.853326    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:23.854962 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.854942    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 27 19:29:23.855091 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.855038    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 27 19:29:23.855176 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.855123    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.855256 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.855189    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f44zk\""
Jul 27 19:29:23.855317 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.855307    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.855468 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.855414    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 27 19:29:23.855983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.855966    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 27 19:29:23.856225 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.856209    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nht7b\""
Jul 27 19:29:23.856408 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.856358    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 19:29:23.856408 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.856387    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"]
Jul 27 19:29:23.856543 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.856442    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nxgrc"]
Jul 27 19:29:23.856596 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.856565    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 19:29:23.856663 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.856632    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:23.858947 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.858931    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 19:29:23.859117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.859031    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-z4xzl\""
Jul 27 19:29:23.859529 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.859506    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"]
Jul 27 19:29:23.860651 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.860628    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 27 19:29:23.860877 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.860860    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.861013 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.860919    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:23.861341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.860985    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.862887 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.862869    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"]
Jul 27 19:29:23.863006 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.862986    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"
Jul 27 19:29:23.863423 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.863405    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 19:29:23.863713 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.863698    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mdj99\""
Jul 27 19:29:23.863825 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.863808    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 19:29:23.863910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.863783    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 19:29:23.865259 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.865236    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gk6qn\""
Jul 27 19:29:23.865408 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.865310    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"]
Jul 27 19:29:23.865564 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.865547    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:23.868303 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868026    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 27 19:29:23.868418 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868396    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-62tqr"]
Jul 27 19:29:23.868480 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868430    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"]
Jul 27 19:29:23.868480 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868449    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75ccb94d97-g2mbc"]
Jul 27 19:29:23.868480 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868467    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zqrl2"]
Jul 27 19:29:23.868578 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868488    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nxgrc"]
Jul 27 19:29:23.868578 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868507    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"]
Jul 27 19:29:23.868578 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.868524    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.869166 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.869153    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"]
Jul 27 19:29:23.870500 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.870180    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"]
Jul 27 19:29:23.871165 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.871149    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"]
Jul 27 19:29:23.872120 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.872101    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"]
Jul 27 19:29:23.872519 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.872502    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 27 19:29:23.872676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.872654    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 27 19:29:23.872929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.872909    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 27 19:29:23.873017 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.872598    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 27 19:29:23.874280 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.874260    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"]
Jul 27 19:29:23.878903 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.878885    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"]
Jul 27 19:29:23.881209 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.881186    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-645bb56f9f-b8xdb"]
Jul 27 19:29:23.919184 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919164    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4fceebef-ef4a-4f7d-af19-a773ba51379f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:23.919316 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919192    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-installation-pull-secrets\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.919316 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919215    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlntq\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-kube-api-access-jlntq\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.919316 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919270    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.919316 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919305    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919331    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bd65d64-fb4f-411e-a882-80a318f2f24b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-787455f47d-plsbt\" (UID: \"0bd65d64-fb4f-411e-a882-80a318f2f24b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919353    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfbcdbee-4f5a-49ca-bb43-157da996dad9-serving-cert\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919380    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2fdf52d3-779d-440d-aace-29b908c7e658-config\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919400    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1202964d-8089-48b2-8302-6c4d64a4357d-tmp\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919423    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-certificates\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919445    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-bound-sa-token\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.919498 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919466    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-stats-auth\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919498    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rmg\" (UniqueName: \"kubernetes.io/projected/4fceebef-ef4a-4f7d-af19-a773ba51379f-kube-api-access-94rmg\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919523    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919570    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49r6\" (UniqueName: \"kubernetes.io/projected/64904d6b-c545-4604-88e5-b7a5d38a4d37-kube-api-access-x49r6\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919606    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt5zl\" (UniqueName: \"kubernetes.io/projected/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-kube-api-access-lt5zl\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919635    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-tmp\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919661    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zznw8\" (UniqueName: \"kubernetes.io/projected/2fdf52d3-779d-440d-aace-29b908c7e658-kube-api-access-zznw8\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919686    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56j57\" (UniqueName: \"kubernetes.io/projected/0af76269-e664-48e9-addb-350ebc791abd-kube-api-access-56j57\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919711    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919755    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nwz8\" (UniqueName: \"kubernetes.io/projected/07f8c00f-e1ad-4805-8ec3-3465493eddca-kube-api-access-5nwz8\") pod \"ingress-canary-nxgrc\" (UID: \"07f8c00f-e1ad-4805-8ec3-3465493eddca\") " pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919778    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919803    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f8c00f-e1ad-4805-8ec3-3465493eddca-cert\") pod \"ingress-canary-nxgrc\" (UID: \"07f8c00f-e1ad-4805-8ec3-3465493eddca\") " pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:23.919846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919836    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-hub\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919889    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbl85\" (UniqueName: \"kubernetes.io/projected/bfbcdbee-4f5a-49ca-bb43-157da996dad9-kube-api-access-kbl85\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919915    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919941    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.919963    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-tmp-dir\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920010    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920039    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-ca-trust-extracted\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920074    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dt6z\" (UniqueName: \"kubernetes.io/projected/2e0d2ae6-7377-4a43-8430-3b65150f65af-kube-api-access-2dt6z\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920126    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1202964d-8089-48b2-8302-6c4d64a4357d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920160    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/720c7565-d143-44b1-9fae-35306effed02-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920184    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx2xh\" (UniqueName: \"kubernetes.io/projected/ec93e577-f2bf-4848-937c-2a5dbe08eeff-kube-api-access-nx2xh\") pod \"network-check-source-7d88cb55d8-f6sz9\" (UID: \"ec93e577-f2bf-4848-937c-2a5dbe08eeff\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"
Jul 27 19:29:23.920288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920224    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvvq7\" (UniqueName: \"kubernetes.io/projected/720c7565-d143-44b1-9fae-35306effed02-kube-api-access-qvvq7\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.920670 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.920246    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-trusted-ca\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921304    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfbcdbee-4f5a-49ca-bb43-157da996dad9-trusted-ca\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921358    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-config-volume\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921393    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-default-certificate\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921425    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921450    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921481    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921509    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0af76269-e664-48e9-addb-350ebc791abd-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921552    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-snapshots\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921579    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8hc8\" (UniqueName: \"kubernetes.io/projected/0bd65d64-fb4f-411e-a882-80a318f2f24b-kube-api-access-j8hc8\") pod \"managed-serviceaccount-addon-agent-787455f47d-plsbt\" (UID: \"0bd65d64-fb4f-411e-a882-80a318f2f24b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921608    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2fdf52d3-779d-440d-aace-29b908c7e658-serving-cert\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921642    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-image-registry-private-configuration\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921676    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-serving-cert\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921710    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0af76269-e664-48e9-addb-350ebc791abd-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921752    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-service-ca-bundle\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.923930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921777    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4qlj\" (UniqueName: \"kubernetes.io/projected/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-kube-api-access-l4qlj\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:23.924630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921806    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwbnn\" (UniqueName: \"kubernetes.io/projected/1202964d-8089-48b2-8302-6c4d64a4357d-kube-api-access-fwbnn\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:23.924630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921854    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-ca\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:23.924630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921883    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tndn\" (UniqueName: \"kubernetes.io/projected/ee3cc093-676d-4825-b54b-0a95e073bde6-kube-api-access-8tndn\") pod \"volume-data-source-validator-549c6d54cf-92c5x\" (UID: \"ee3cc093-676d-4825-b54b-0a95e073bde6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"
Jul 27 19:29:23.924630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:23.921906    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bfbcdbee-4f5a-49ca-bb43-157da996dad9-config\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.022625 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022590    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-certificates\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.022843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022643    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-bound-sa-token\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.022843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022675    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-stats-auth\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.022843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022724    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rmg\" (UniqueName: \"kubernetes.io/projected/4fceebef-ef4a-4f7d-af19-a773ba51379f-kube-api-access-94rmg\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:24.022843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022783    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:24.022843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022805    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49r6\" (UniqueName: \"kubernetes.io/projected/64904d6b-c545-4604-88e5-b7a5d38a4d37-kube-api-access-x49r6\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:24.022843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022826    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt5zl\" (UniqueName: \"kubernetes.io/projected/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-kube-api-access-lt5zl\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022850    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-tmp\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022872    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zznw8\" (UniqueName: \"kubernetes.io/projected/2fdf52d3-779d-440d-aace-29b908c7e658-kube-api-access-zznw8\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022898    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56j57\" (UniqueName: \"kubernetes.io/projected/0af76269-e664-48e9-addb-350ebc791abd-kube-api-access-56j57\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022922    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022945    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nwz8\" (UniqueName: \"kubernetes.io/projected/07f8c00f-e1ad-4805-8ec3-3465493eddca-kube-api-access-5nwz8\") pod \"ingress-canary-nxgrc\" (UID: \"07f8c00f-e1ad-4805-8ec3-3465493eddca\") " pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.022965    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.023144 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.023104    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.023173    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls podName:64904d6b-c545-4604-88e5-b7a5d38a4d37 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.523151052 +0000 UTC m=+183.180860577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls") pod "cluster-samples-operator-6748489656-phbk2" (UID: "64904d6b-c545-4604-88e5-b7a5d38a4d37") : secret "samples-operator-tls" not found
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.023198    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.023261    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.523243343 +0000 UTC m=+183.180952862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : secret "router-metrics-certs-default" not found
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023315    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-tmp\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023388    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f8c00f-e1ad-4805-8ec3-3465493eddca-cert\") pod \"ingress-canary-nxgrc\" (UID: \"07f8c00f-e1ad-4805-8ec3-3465493eddca\") " pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023404    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-certificates\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.023453 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023422    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-hub\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023471    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbl85\" (UniqueName: \"kubernetes.io/projected/bfbcdbee-4f5a-49ca-bb43-157da996dad9-kube-api-access-kbl85\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.023491    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.52347614 +0000 UTC m=+183.181185666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : configmap references non-existent config key: service-ca.crt
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023524    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023552    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023578    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-tmp-dir\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023605    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023629    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-ca-trust-extracted\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023653    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dt6z\" (UniqueName: \"kubernetes.io/projected/2e0d2ae6-7377-4a43-8430-3b65150f65af-kube-api-access-2dt6z\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023682    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1202964d-8089-48b2-8302-6c4d64a4357d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023710    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/720c7565-d143-44b1-9fae-35306effed02-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023754    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx2xh\" (UniqueName: \"kubernetes.io/projected/ec93e577-f2bf-4848-937c-2a5dbe08eeff-kube-api-access-nx2xh\") pod \"network-check-source-7d88cb55d8-f6sz9\" (UID: \"ec93e577-f2bf-4848-937c-2a5dbe08eeff\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"
Jul 27 19:29:24.023821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023796    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvvq7\" (UniqueName: \"kubernetes.io/projected/720c7565-d143-44b1-9fae-35306effed02-kube-api-access-qvvq7\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023828    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-trusted-ca\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023852    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfbcdbee-4f5a-49ca-bb43-157da996dad9-trusted-ca\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023876    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-config-volume\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023902    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-default-certificate\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023930    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023952    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023978    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.023974    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-tmp-dir\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024003    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0af76269-e664-48e9-addb-350ebc791abd-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024043    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-snapshots\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.024085    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024091    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8hc8\" (UniqueName: \"kubernetes.io/projected/0bd65d64-fb4f-411e-a882-80a318f2f24b-kube-api-access-j8hc8\") pod \"managed-serviceaccount-addon-agent-787455f47d-plsbt\" (UID: \"0bd65d64-fb4f-411e-a882-80a318f2f24b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024234    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:24.024400 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.024333    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024421    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-ca-trust-extracted\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.024131    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert podName:9fd028b2-a23b-416a-92bc-2f7e15ae1ae7 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.524114935 +0000 UTC m=+183.181824456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4hhz6" (UID: "9fd028b2-a23b-416a-92bc-2f7e15ae1ae7") : secret "networking-console-plugin-cert" not found
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.024563    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls podName:4fceebef-ef4a-4f7d-af19-a773ba51379f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.524544824 +0000 UTC m=+183.182254347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6nhm9" (UID: "4fceebef-ef4a-4f7d-af19-a773ba51379f") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024592    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2fdf52d3-779d-440d-aace-29b908c7e658-serving-cert\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024640    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-image-registry-private-configuration\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024678    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-serving-cert\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024719    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0af76269-e664-48e9-addb-350ebc791abd-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024765    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-service-ca-bundle\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024795    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4qlj\" (UniqueName: \"kubernetes.io/projected/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-kube-api-access-l4qlj\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024828    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwbnn\" (UniqueName: \"kubernetes.io/projected/1202964d-8089-48b2-8302-6c4d64a4357d-kube-api-access-fwbnn\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024866    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-ca\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024894    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tndn\" (UniqueName: \"kubernetes.io/projected/ee3cc093-676d-4825-b54b-0a95e073bde6-kube-api-access-8tndn\") pod \"volume-data-source-validator-549c6d54cf-92c5x\" (UID: \"ee3cc093-676d-4825-b54b-0a95e073bde6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024899    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-snapshots\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.025117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024920    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bfbcdbee-4f5a-49ca-bb43-157da996dad9-config\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024953    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4fceebef-ef4a-4f7d-af19-a773ba51379f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.024980    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-installation-pull-secrets\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025008    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlntq\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-kube-api-access-jlntq\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025041    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025067    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025092    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bd65d64-fb4f-411e-a882-80a318f2f24b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-787455f47d-plsbt\" (UID: \"0bd65d64-fb4f-411e-a882-80a318f2f24b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025117    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfbcdbee-4f5a-49ca-bb43-157da996dad9-serving-cert\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025143    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2fdf52d3-779d-440d-aace-29b908c7e658-config\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025169    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1202964d-8089-48b2-8302-6c4d64a4357d-tmp\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025329    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/720c7565-d143-44b1-9fae-35306effed02-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.025432    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.025446    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75ccb94d97-g2mbc: secret "image-registry-tls" not found
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025449    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-config-volume\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.025488    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls podName:c6c6e6e0-40d7-46cc-8d0a-bbec77770677 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.525475125 +0000 UTC m=+183.183184634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls") pod "image-registry-75ccb94d97-g2mbc" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677") : secret "image-registry-tls" not found
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.025565    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.025841 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.025596    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls podName:2f225d20-1fb1-4cd0-bf76-31d0b51f791f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:24.525585674 +0000 UTC m=+183.183295182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls") pod "dns-default-zqrl2" (UID: "2f225d20-1fb1-4cd0-bf76-31d0b51f791f") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.026657 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.025599    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1202964d-8089-48b2-8302-6c4d64a4357d-tmp\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.026657 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.026001    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4fceebef-ef4a-4f7d-af19-a773ba51379f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:24.026657 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.026414    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-hub\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.026657 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.026448    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-service-ca-bundle\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.027297    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-trusted-ca\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.027717    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-stats-auth\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028115    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfbcdbee-4f5a-49ca-bb43-157da996dad9-trusted-ca\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028146    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-default-certificate\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028174    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07f8c00f-e1ad-4805-8ec3-3465493eddca-cert\") pod \"ingress-canary-nxgrc\" (UID: \"07f8c00f-e1ad-4805-8ec3-3465493eddca\") " pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028457    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028701    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0af76269-e664-48e9-addb-350ebc791abd-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028803    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0af76269-e664-48e9-addb-350ebc791abd-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.029085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.028962    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.029681 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.029658    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-ca\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.029839 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.029749    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-installation-pull-secrets\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.030255 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.030226    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2fdf52d3-779d-440d-aace-29b908c7e658-config\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.030912 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.030888    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bfbcdbee-4f5a-49ca-bb43-157da996dad9-config\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.031026 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.030995    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfbcdbee-4f5a-49ca-bb43-157da996dad9-serving-cert\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.032180 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.031662    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/720c7565-d143-44b1-9fae-35306effed02-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.032180 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.032018    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bd65d64-fb4f-411e-a882-80a318f2f24b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-787455f47d-plsbt\" (UID: \"0bd65d64-fb4f-411e-a882-80a318f2f24b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:24.033059 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.032583    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1202964d-8089-48b2-8302-6c4d64a4357d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.033059 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.032691    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-serving-cert\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.034255 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.033602    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt5zl\" (UniqueName: \"kubernetes.io/projected/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-kube-api-access-lt5zl\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.034255 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.033690    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nwz8\" (UniqueName: \"kubernetes.io/projected/07f8c00f-e1ad-4805-8ec3-3465493eddca-kube-api-access-5nwz8\") pod \"ingress-canary-nxgrc\" (UID: \"07f8c00f-e1ad-4805-8ec3-3465493eddca\") " pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:24.034255 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.033965    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbl85\" (UniqueName: \"kubernetes.io/projected/bfbcdbee-4f5a-49ca-bb43-157da996dad9-kube-api-access-kbl85\") pod \"console-operator-584dcb6b54-dwtww\" (UID: \"bfbcdbee-4f5a-49ca-bb43-157da996dad9\") " pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.035124 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.034809    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dt6z\" (UniqueName: \"kubernetes.io/projected/2e0d2ae6-7377-4a43-8430-3b65150f65af-kube-api-access-2dt6z\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.035265 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.035235    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56j57\" (UniqueName: \"kubernetes.io/projected/0af76269-e664-48e9-addb-350ebc791abd-kube-api-access-56j57\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5\" (UID: \"0af76269-e664-48e9-addb-350ebc791abd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.035801 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.035479    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8hc8\" (UniqueName: \"kubernetes.io/projected/0bd65d64-fb4f-411e-a882-80a318f2f24b-kube-api-access-j8hc8\") pod \"managed-serviceaccount-addon-agent-787455f47d-plsbt\" (UID: \"0bd65d64-fb4f-411e-a882-80a318f2f24b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:24.035899 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.035859    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-image-registry-private-configuration\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.036398 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.036342    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwbnn\" (UniqueName: \"kubernetes.io/projected/1202964d-8089-48b2-8302-6c4d64a4357d-kube-api-access-fwbnn\") pod \"klusterlet-addon-workmgr-7566d7c47d-zbbjm\" (UID: \"1202964d-8089-48b2-8302-6c4d64a4357d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.037816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.037478    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-bound-sa-token\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.037816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.037714    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2fdf52d3-779d-440d-aace-29b908c7e658-serving-cert\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.038073 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.038045    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx2xh\" (UniqueName: \"kubernetes.io/projected/ec93e577-f2bf-4848-937c-2a5dbe08eeff-kube-api-access-nx2xh\") pod \"network-check-source-7d88cb55d8-f6sz9\" (UID: \"ec93e577-f2bf-4848-937c-2a5dbe08eeff\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"
Jul 27 19:29:24.038163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.038138    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zznw8\" (UniqueName: \"kubernetes.io/projected/2fdf52d3-779d-440d-aace-29b908c7e658-kube-api-access-zznw8\") pod \"service-ca-operator-5d6c75f84f-5qljl\" (UID: \"2fdf52d3-779d-440d-aace-29b908c7e658\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.038743 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.038710    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49r6\" (UniqueName: \"kubernetes.io/projected/64904d6b-c545-4604-88e5-b7a5d38a4d37-kube-api-access-x49r6\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:24.039040 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.039021    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4qlj\" (UniqueName: \"kubernetes.io/projected/c6b55b08-2fc0-4416-bba2-bd4f0d5f6308-kube-api-access-l4qlj\") pod \"insights-operator-5885fdb759-62tqr\" (UID: \"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308\") " pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.039231 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.039218    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rmg\" (UniqueName: \"kubernetes.io/projected/4fceebef-ef4a-4f7d-af19-a773ba51379f-kube-api-access-94rmg\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:24.039362 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.039343    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvvq7\" (UniqueName: \"kubernetes.io/projected/720c7565-d143-44b1-9fae-35306effed02-kube-api-access-qvvq7\") pod \"cluster-proxy-proxy-agent-bc9f4ff98-59bt6\" (UID: \"720c7565-d143-44b1-9fae-35306effed02\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.045461 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.045440    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tndn\" (UniqueName: \"kubernetes.io/projected/ee3cc093-676d-4825-b54b-0a95e073bde6-kube-api-access-8tndn\") pod \"volume-data-source-validator-549c6d54cf-92c5x\" (UID: \"ee3cc093-676d-4825-b54b-0a95e073bde6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"
Jul 27 19:29:24.045586 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.045565    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlntq\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-kube-api-access-jlntq\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.133411 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.133369    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:24.154975 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.154949    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"
Jul 27 19:29:24.187557 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.187319    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-62tqr"
Jul 27 19:29:24.193092 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.192888    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"
Jul 27 19:29:24.198158 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.197950    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"
Jul 27 19:29:24.229247 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.228451    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"
Jul 27 19:29:24.234337 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.234132    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nxgrc"
Jul 27 19:29:24.241895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.240833    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"
Jul 27 19:29:24.245485 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.245137    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:24.250071 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.249909    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:29:24.306753 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.304490    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-dwtww"]
Jul 27 19:29:24.321912 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.321867    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfbcdbee_4f5a_49ca_bb43_157da996dad9.slice/crio-1141802bbc6f1904059a1b61ba947e92e08d18e72254358a505b6fa56c0117b3 WatchSource:0}: Error finding container 1141802bbc6f1904059a1b61ba947e92e08d18e72254358a505b6fa56c0117b3: Status 404 returned error can't find the container with id 1141802bbc6f1904059a1b61ba947e92e08d18e72254358a505b6fa56c0117b3
Jul 27 19:29:24.339241 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.338506    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5"]
Jul 27 19:29:24.383604 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.383468    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" event={"ID":"bfbcdbee-4f5a-49ca-bb43-157da996dad9","Type":"ContainerStarted","Data":"1141802bbc6f1904059a1b61ba947e92e08d18e72254358a505b6fa56c0117b3"}
Jul 27 19:29:24.385160 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.385097    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5" event={"ID":"0af76269-e664-48e9-addb-350ebc791abd","Type":"ContainerStarted","Data":"e467589b441807815db7305d6d9445a80ecade795a738fb10dd4e41fac7441f5"}
Jul 27 19:29:24.467011 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.466868    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x"]
Jul 27 19:29:24.532586 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532554    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:24.532586 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532589    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532649    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532695    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532700    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532717    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75ccb94d97-g2mbc: secret "image-registry-tls" not found
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532720    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532776    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532791    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.532827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532800    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls podName:c6c6e6e0-40d7-46cc-8d0a-bbec77770677 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.532783355 +0000 UTC m=+184.190492865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls") pod "image-registry-75ccb94d97-g2mbc" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677") : secret "image-registry-tls" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532828    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.532841    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532848    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls podName:2f225d20-1fb1-4cd0-bf76-31d0b51f791f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.532831432 +0000 UTC m=+184.190540961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls") pod "dns-default-zqrl2" (UID: "2f225d20-1fb1-4cd0-bf76-31d0b51f791f") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532873    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.532863665 +0000 UTC m=+184.190573178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : configmap references non-existent config key: service-ca.crt
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532896    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.532880495 +0000 UTC m=+184.190590054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : secret "router-metrics-certs-default" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532834    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532796    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532944    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.532981    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls podName:64904d6b-c545-4604-88e5-b7a5d38a4d37 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.532964261 +0000 UTC m=+184.190673771 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls") pod "cluster-samples-operator-6748489656-phbk2" (UID: "64904d6b-c545-4604-88e5-b7a5d38a4d37") : secret "samples-operator-tls" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.533003    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls podName:4fceebef-ef4a-4f7d-af19-a773ba51379f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.532990255 +0000 UTC m=+184.190699766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6nhm9" (UID: "4fceebef-ef4a-4f7d-af19-a773ba51379f") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:24.533141 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:24.533018    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert podName:9fd028b2-a23b-416a-92bc-2f7e15ae1ae7 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:25.533010457 +0000 UTC m=+184.190719971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4hhz6" (UID: "9fd028b2-a23b-416a-92bc-2f7e15ae1ae7") : secret "networking-console-plugin-cert" not found
Jul 27 19:29:24.543540 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.543519    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"]
Jul 27 19:29:24.546374 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.546351    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1202964d_8089_48b2_8302_6c4d64a4357d.slice/crio-2469a5774f369987fa2cd368787f29fd4fc2892e08fe5271d1ee86d06039734d WatchSource:0}: Error finding container 2469a5774f369987fa2cd368787f29fd4fc2892e08fe5271d1ee86d06039734d: Status 404 returned error can't find the container with id 2469a5774f369987fa2cd368787f29fd4fc2892e08fe5271d1ee86d06039734d
Jul 27 19:29:24.660007 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.659940    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl"]
Jul 27 19:29:24.663063 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.663033    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-62tqr"]
Jul 27 19:29:24.663298 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.663279    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdf52d3_779d_440d_aace_29b908c7e658.slice/crio-8af20c317e95540212e71d6b6e36e46bfd83c65be3fb6ad5c92cb336f45f400c WatchSource:0}: Error finding container 8af20c317e95540212e71d6b6e36e46bfd83c65be3fb6ad5c92cb336f45f400c: Status 404 returned error can't find the container with id 8af20c317e95540212e71d6b6e36e46bfd83c65be3fb6ad5c92cb336f45f400c
Jul 27 19:29:24.666046 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.666027    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b55b08_2fc0_4416_bba2_bd4f0d5f6308.slice/crio-d2248935042f865ebafc8d1af25e70779fb5a05acdc006856179b4a59669dd35 WatchSource:0}: Error finding container d2248935042f865ebafc8d1af25e70779fb5a05acdc006856179b4a59669dd35: Status 404 returned error can't find the container with id d2248935042f865ebafc8d1af25e70779fb5a05acdc006856179b4a59669dd35
Jul 27 19:29:24.703931 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.703902    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9"]
Jul 27 19:29:24.707429 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.707392    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec93e577_f2bf_4848_937c_2a5dbe08eeff.slice/crio-601553c35e8c679f57a56530348972768cd039d5b8346e3adf22570823d3b7ee WatchSource:0}: Error finding container 601553c35e8c679f57a56530348972768cd039d5b8346e3adf22570823d3b7ee: Status 404 returned error can't find the container with id 601553c35e8c679f57a56530348972768cd039d5b8346e3adf22570823d3b7ee
Jul 27 19:29:24.707561 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.707428    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nxgrc"]
Jul 27 19:29:24.708814 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.708723    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt"]
Jul 27 19:29:24.709973 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:24.709918    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"]
Jul 27 19:29:24.711171 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.711150    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bd65d64_fb4f_411e_a882_80a318f2f24b.slice/crio-94ece0d81fdc29f2fa68316d86d1dbea5fe80c4b849e9ea83ebbf237a67e9614 WatchSource:0}: Error finding container 94ece0d81fdc29f2fa68316d86d1dbea5fe80c4b849e9ea83ebbf237a67e9614: Status 404 returned error can't find the container with id 94ece0d81fdc29f2fa68316d86d1dbea5fe80c4b849e9ea83ebbf237a67e9614
Jul 27 19:29:24.712226 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.712050    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod720c7565_d143_44b1_9fae_35306effed02.slice/crio-72cc8585ab8b2141eb62ee7d72c3c513fe71e067bc1da34eb7623a41b6f89e8c WatchSource:0}: Error finding container 72cc8585ab8b2141eb62ee7d72c3c513fe71e067bc1da34eb7623a41b6f89e8c: Status 404 returned error can't find the container with id 72cc8585ab8b2141eb62ee7d72c3c513fe71e067bc1da34eb7623a41b6f89e8c
Jul 27 19:29:24.712787 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:24.712764    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f8c00f_e1ad_4805_8ec3_3465493eddca.slice/crio-5468ebaef2389bb916cf56ce98f9c42899629912ad648336d9febb7ea784bb76 WatchSource:0}: Error finding container 5468ebaef2389bb916cf56ce98f9c42899629912ad648336d9febb7ea784bb76: Status 404 returned error can't find the container with id 5468ebaef2389bb916cf56ce98f9c42899629912ad648336d9febb7ea784bb76
Jul 27 19:29:25.392582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.391801    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9" event={"ID":"ec93e577-f2bf-4848-937c-2a5dbe08eeff","Type":"ContainerStarted","Data":"e02436952c355786a3997ac4cd403ea633591b4f135fed975187c99928a0a2b3"}
Jul 27 19:29:25.392582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.391867    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9" event={"ID":"ec93e577-f2bf-4848-937c-2a5dbe08eeff","Type":"ContainerStarted","Data":"601553c35e8c679f57a56530348972768cd039d5b8346e3adf22570823d3b7ee"}
Jul 27 19:29:25.399988 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.399911    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nxgrc" event={"ID":"07f8c00f-e1ad-4805-8ec3-3465493eddca","Type":"ContainerStarted","Data":"5468ebaef2389bb916cf56ce98f9c42899629912ad648336d9febb7ea784bb76"}
Jul 27 19:29:25.403552 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.403423    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt" event={"ID":"0bd65d64-fb4f-411e-a882-80a318f2f24b","Type":"ContainerStarted","Data":"94ece0d81fdc29f2fa68316d86d1dbea5fe80c4b849e9ea83ebbf237a67e9614"}
Jul 27 19:29:25.411775 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.411428    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-f6sz9" podStartSLOduration=117.41140893 podStartE2EDuration="1m57.41140893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:25.409578805 +0000 UTC m=+184.067288334" watchObservedRunningTime="2026-07-27 19:29:25.41140893 +0000 UTC m=+184.069118459"
Jul 27 19:29:25.411775 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.411704    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" event={"ID":"720c7565-d143-44b1-9fae-35306effed02","Type":"ContainerStarted","Data":"72cc8585ab8b2141eb62ee7d72c3c513fe71e067bc1da34eb7623a41b6f89e8c"}
Jul 27 19:29:25.414415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.414333    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x" event={"ID":"ee3cc093-676d-4825-b54b-0a95e073bde6","Type":"ContainerStarted","Data":"9570b45478f88ea70274634ed872ec58e3ddacf426b36e4ad727e7a76798d30c"}
Jul 27 19:29:25.416450 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.416364    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl" event={"ID":"2fdf52d3-779d-440d-aace-29b908c7e658","Type":"ContainerStarted","Data":"8af20c317e95540212e71d6b6e36e46bfd83c65be3fb6ad5c92cb336f45f400c"}
Jul 27 19:29:25.419465 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.419414    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-62tqr" event={"ID":"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308","Type":"ContainerStarted","Data":"d2248935042f865ebafc8d1af25e70779fb5a05acdc006856179b4a59669dd35"}
Jul 27 19:29:25.420961 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.420912    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm" event={"ID":"1202964d-8089-48b2-8302-6c4d64a4357d","Type":"ContainerStarted","Data":"2469a5774f369987fa2cd368787f29fd4fc2892e08fe5271d1ee86d06039734d"}
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541511    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541580    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541609    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541648    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541677    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541746    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:25.541777    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.541940    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542003    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls podName:2f225d20-1fb1-4cd0-bf76-31d0b51f791f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.541984388 +0000 UTC m=+186.199693898 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls") pod "dns-default-zqrl2" (UID: "2f225d20-1fb1-4cd0-bf76-31d0b51f791f") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542401    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542450    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls podName:64904d6b-c545-4604-88e5-b7a5d38a4d37 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.542434435 +0000 UTC m=+186.200143943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls") pod "cluster-samples-operator-6748489656-phbk2" (UID: "64904d6b-c545-4604-88e5-b7a5d38a4d37") : secret "samples-operator-tls" not found
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542506    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542537    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.54252735 +0000 UTC m=+186.200236860 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : secret "router-metrics-certs-default" not found
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542615    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.542605415 +0000 UTC m=+186.200314926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : configmap references non-existent config key: service-ca.crt
Jul 27 19:29:25.542937 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542667    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:25.543855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542696    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls podName:4fceebef-ef4a-4f7d-af19-a773ba51379f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.542686683 +0000 UTC m=+186.200396193 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6nhm9" (UID: "4fceebef-ef4a-4f7d-af19-a773ba51379f") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:25.543855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542761    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 19:29:25.543855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542793    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert podName:9fd028b2-a23b-416a-92bc-2f7e15ae1ae7 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.542780998 +0000 UTC m=+186.200490510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4hhz6" (UID: "9fd028b2-a23b-416a-92bc-2f7e15ae1ae7") : secret "networking-console-plugin-cert" not found
Jul 27 19:29:25.543855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542854    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:25.543855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542865    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75ccb94d97-g2mbc: secret "image-registry-tls" not found
Jul 27 19:29:25.543855 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:25.542894    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls podName:c6c6e6e0-40d7-46cc-8d0a-bbec77770677 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:27.542884029 +0000 UTC m=+186.200593540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls") pod "image-registry-75ccb94d97-g2mbc" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677") : secret "image-registry-tls" not found
Jul 27 19:29:26.460435 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:26.460363    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x" event={"ID":"ee3cc093-676d-4825-b54b-0a95e073bde6","Type":"ContainerStarted","Data":"a63d8fefdf5f64653f5416c34a3c7d599a36ec9c101dbd9600c4f045fda16d03"}
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564054    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564113    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564147    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564183    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564212    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564268    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:27.564302    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564428    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.564409135 +0000 UTC m=+190.222118663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : configmap references non-existent config key: service-ca.crt
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564428    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564473    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert podName:9fd028b2-a23b-416a-92bc-2f7e15ae1ae7 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.564464433 +0000 UTC m=+190.222173956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4hhz6" (UID: "9fd028b2-a23b-416a-92bc-2f7e15ae1ae7") : secret "networking-console-plugin-cert" not found
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564501    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564530    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564563    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls podName:4fceebef-ef4a-4f7d-af19-a773ba51379f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.564545574 +0000 UTC m=+190.222255085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6nhm9" (UID: "4fceebef-ef4a-4f7d-af19-a773ba51379f") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564582    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls podName:64904d6b-c545-4604-88e5-b7a5d38a4d37 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.564572877 +0000 UTC m=+190.222282390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls") pod "cluster-samples-operator-6748489656-phbk2" (UID: "64904d6b-c545-4604-88e5-b7a5d38a4d37") : secret "samples-operator-tls" not found
Jul 27 19:29:27.564760 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564600    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:29:27.565938 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564634    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.564621269 +0000 UTC m=+190.222330779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : secret "router-metrics-certs-default" not found
Jul 27 19:29:27.565938 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564711    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:27.565938 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564722    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75ccb94d97-g2mbc: secret "image-registry-tls" not found
Jul 27 19:29:27.565938 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.564774    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls podName:c6c6e6e0-40d7-46cc-8d0a-bbec77770677 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.564762725 +0000 UTC m=+190.222472234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls") pod "image-registry-75ccb94d97-g2mbc" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677") : secret "image-registry-tls" not found
Jul 27 19:29:27.566146 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.566125    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:27.566206 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:27.566183    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls podName:2f225d20-1fb1-4cd0-bf76-31d0b51f791f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:31.566169038 +0000 UTC m=+190.223878547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls") pod "dns-default-zqrl2" (UID: "2f225d20-1fb1-4cd0-bf76-31d0b51f791f") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:31.608210 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608171    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608220    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608257    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608290    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608342    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608367    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608395    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608421    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608447    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.608427863 +0000 UTC m=+198.266137371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : configmap references non-existent config key: service-ca.crt
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608454    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608500    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.608486138 +0000 UTC m=+198.266195646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : secret "router-metrics-certs-default" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608473    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608514    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608531    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75ccb94d97-g2mbc: secret "image-registry-tls" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608546    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert podName:9fd028b2-a23b-416a-92bc-2f7e15ae1ae7 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.60853718 +0000 UTC m=+198.266246692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4hhz6" (UID: "9fd028b2-a23b-416a-92bc-2f7e15ae1ae7") : secret "networking-console-plugin-cert" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608520    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608562    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls podName:4fceebef-ef4a-4f7d-af19-a773ba51379f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.608554631 +0000 UTC m=+198.266264139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6nhm9" (UID: "4fceebef-ef4a-4f7d-af19-a773ba51379f") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:31.608827 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608568    2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 19:29:31.609687 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608579    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls podName:c6c6e6e0-40d7-46cc-8d0a-bbec77770677 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.608571181 +0000 UTC m=+198.266280700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls") pod "image-registry-75ccb94d97-g2mbc" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677") : secret "image-registry-tls" not found
Jul 27 19:29:31.609687 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608613    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls podName:2f225d20-1fb1-4cd0-bf76-31d0b51f791f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.608587993 +0000 UTC m=+198.266297502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls") pod "dns-default-zqrl2" (UID: "2f225d20-1fb1-4cd0-bf76-31d0b51f791f") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:31.609687 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608629    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs podName:f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01 nodeName:}" failed. No retries permitted until 2026-07-27 19:31:33.608621623 +0000 UTC m=+312.266331138 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs") pod "network-metrics-daemon-7d4b9" (UID: "f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01") : secret "metrics-daemon-secret" not found
Jul 27 19:29:31.609687 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:31.608723    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:31.609687 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608837    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:29:31.609687 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:31.608886    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls podName:64904d6b-c545-4604-88e5-b7a5d38a4d37 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.608868878 +0000 UTC m=+198.266578387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls") pod "cluster-samples-operator-6748489656-phbk2" (UID: "64904d6b-c545-4604-88e5-b7a5d38a4d37") : secret "samples-operator-tls" not found
Jul 27 19:29:34.489213 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.489128    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/0.log"
Jul 27 19:29:34.489213 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.489179    2544 generic.go:356] "Generic (PLEG): container finished" podID="bfbcdbee-4f5a-49ca-bb43-157da996dad9" containerID="d208656b03e7b1c3e1c2d6941307b0d4f4aec595c78c74a725c0db75fef357c0" exitCode=255
Jul 27 19:29:34.489717 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.489272    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" event={"ID":"bfbcdbee-4f5a-49ca-bb43-157da996dad9","Type":"ContainerDied","Data":"d208656b03e7b1c3e1c2d6941307b0d4f4aec595c78c74a725c0db75fef357c0"}
Jul 27 19:29:34.489717 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.489491    2544 scope.go:117] "RemoveContainer" containerID="d208656b03e7b1c3e1c2d6941307b0d4f4aec595c78c74a725c0db75fef357c0"
Jul 27 19:29:34.490927 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.490903    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5" event={"ID":"0af76269-e664-48e9-addb-350ebc791abd","Type":"ContainerStarted","Data":"20c889bae0e9f458b87e5868f08e0d12d1251e245c186e4b8fffb7faddc0dad2"}
Jul 27 19:29:34.493006 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.492983    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nxgrc" event={"ID":"07f8c00f-e1ad-4805-8ec3-3465493eddca","Type":"ContainerStarted","Data":"9851d1ca73ce0128e26a7b9d19a28656bcf8568efc0012218c2d58f4daff59e0"}
Jul 27 19:29:34.494506 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.494477    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt" event={"ID":"0bd65d64-fb4f-411e-a882-80a318f2f24b","Type":"ContainerStarted","Data":"e41e56994672b96a4593bdb766e26846f83e1e0a73ba0ecf5fcb0b03170c28c9"}
Jul 27 19:29:34.496683 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.496659    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" event={"ID":"720c7565-d143-44b1-9fae-35306effed02","Type":"ContainerStarted","Data":"77671cc898a2b202d61514a849b5e875c48308a6a4386c7d28424a3a75a5d892"}
Jul 27 19:29:34.496778 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.496698    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" event={"ID":"720c7565-d143-44b1-9fae-35306effed02","Type":"ContainerStarted","Data":"17dc8491e2d6cc9a5024a7965d1f089912b681bdfe91dccf9d5e7a3659d67b44"}
Jul 27 19:29:34.496778 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.496715    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" event={"ID":"720c7565-d143-44b1-9fae-35306effed02","Type":"ContainerStarted","Data":"c3ad377f04cfdcee4f826086cbff782805ed84ba1b24e1d78fbf690682d0ebfb"}
Jul 27 19:29:34.498255 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.498230    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl" event={"ID":"2fdf52d3-779d-440d-aace-29b908c7e658","Type":"ContainerStarted","Data":"00d9ba69ced9cba0d802dc7afae87da96ed2dce267abb9dd3818fbc969437f39"}
Jul 27 19:29:34.499885 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.499861    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-62tqr" event={"ID":"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308","Type":"ContainerStarted","Data":"ee861a2c74f0a962aacb7617d63ab6a845031417fb37f378e73d3866f5cda582"}
Jul 27 19:29:34.501205 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.501183    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm" event={"ID":"1202964d-8089-48b2-8302-6c4d64a4357d","Type":"ContainerStarted","Data":"d95158ce3a01dc254d8ecf63e1b379a43370ddf4bb9fe0b971778ca32355ce74"}
Jul 27 19:29:34.501474 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.501457    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:34.503150 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.503133    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm"
Jul 27 19:29:34.510000 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.509923    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-92c5x" podStartSLOduration=124.751042519 podStartE2EDuration="2m6.509906608s" totalImagesPullingTime="1.758864089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:26.479513581 +0000 UTC m=+185.137223111" watchObservedRunningTime="2026-07-27 19:29:34.509906608 +0000 UTC m=+193.167616197"
Jul 27 19:29:34.526392 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.526348    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-62tqr" podStartSLOduration=117.242238315 podStartE2EDuration="2m6.526331077s" totalImagesPullingTime="9.284092762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.525173326 +0000 UTC m=+193.182882853" watchObservedRunningTime="2026-07-27 19:29:34.526331077 +0000 UTC m=+193.184040607"
Jul 27 19:29:34.542257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.540852    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nxgrc" podStartSLOduration=2.302169952 podStartE2EDuration="11.54083462s" totalImagesPullingTime="9.238664668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.540621314 +0000 UTC m=+193.198330846" watchObservedRunningTime="2026-07-27 19:29:34.54083462 +0000 UTC m=+193.198544157"
Jul 27 19:29:34.558333 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.558278    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7566d7c47d-zbbjm" podStartSLOduration=117.136048837 podStartE2EDuration="2m6.558258486s" totalImagesPullingTime="9.422209649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.557209746 +0000 UTC m=+193.214919278" watchObservedRunningTime="2026-07-27 19:29:34.558258486 +0000 UTC m=+193.215968017"
Jul 27 19:29:34.575756 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.575461    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl" podStartSLOduration=117.28942641 podStartE2EDuration="2m6.575443753s" totalImagesPullingTime="9.286017343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.575018337 +0000 UTC m=+193.232727880" watchObservedRunningTime="2026-07-27 19:29:34.575443753 +0000 UTC m=+193.233153282"
Jul 27 19:29:34.594669 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.594622    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" podStartSLOduration=117.308833438 podStartE2EDuration="2m6.594603182s" totalImagesPullingTime="9.285769744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.594162326 +0000 UTC m=+193.251871870" watchObservedRunningTime="2026-07-27 19:29:34.594603182 +0000 UTC m=+193.252312710"
Jul 27 19:29:34.612258 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.611502    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5" podStartSLOduration=117.036929776 podStartE2EDuration="2m6.611480493s" totalImagesPullingTime="9.574550717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.609720207 +0000 UTC m=+193.267429738" watchObservedRunningTime="2026-07-27 19:29:34.611480493 +0000 UTC m=+193.269190026"
Jul 27 19:29:34.625840 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:34.625797    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-787455f47d-plsbt" podStartSLOduration=117.37808196 podStartE2EDuration="2m6.625778147s" totalImagesPullingTime="9.247696187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:34.624824558 +0000 UTC m=+193.282534088" watchObservedRunningTime="2026-07-27 19:29:34.625778147 +0000 UTC m=+193.283487691"
Jul 27 19:29:35.295519 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.295445    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-92r4h"]
Jul 27 19:29:35.297842 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.297820    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.300497 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.300477    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bpnh5\""
Jul 27 19:29:35.300497 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.300489    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 19:29:35.300670 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.300478    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 19:29:35.312077 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.312053    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-92r4h"]
Jul 27 19:29:35.445402 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.445372    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.445575 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.445458    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0209970c-c0d2-4c96-87ef-9e27ffa077b6-crio-socket\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.445575 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.445477    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.445671 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.445610    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0209970c-c0d2-4c96-87ef-9e27ffa077b6-data-volume\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.445719 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.445679    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvrvc\" (UniqueName: \"kubernetes.io/projected/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-api-access-jvrvc\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.505824 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.505796    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:29:35.506226 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.506201    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/0.log"
Jul 27 19:29:35.506279 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.506233    2544 generic.go:356] "Generic (PLEG): container finished" podID="bfbcdbee-4f5a-49ca-bb43-157da996dad9" containerID="f0dc3b2cbbcc3565a27cbba8913aaadc3eb1cf1e6ed68da851b27b0475b23250" exitCode=255
Jul 27 19:29:35.506321 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.506307    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" event={"ID":"bfbcdbee-4f5a-49ca-bb43-157da996dad9","Type":"ContainerDied","Data":"f0dc3b2cbbcc3565a27cbba8913aaadc3eb1cf1e6ed68da851b27b0475b23250"}
Jul 27 19:29:35.506372 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.506349    2544 scope.go:117] "RemoveContainer" containerID="d208656b03e7b1c3e1c2d6941307b0d4f4aec595c78c74a725c0db75fef357c0"
Jul 27 19:29:35.506671 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.506596    2544 scope.go:117] "RemoveContainer" containerID="f0dc3b2cbbcc3565a27cbba8913aaadc3eb1cf1e6ed68da851b27b0475b23250"
Jul 27 19:29:35.506831 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:35.506802    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-dwtww_openshift-console-operator(bfbcdbee-4f5a-49ca-bb43-157da996dad9)\"" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" podUID="bfbcdbee-4f5a-49ca-bb43-157da996dad9"
Jul 27 19:29:35.546950 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.546884    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0209970c-c0d2-4c96-87ef-9e27ffa077b6-data-volume\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.546950 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.546945    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrvc\" (UniqueName: \"kubernetes.io/projected/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-api-access-jvrvc\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.547132 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.546986    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.547132 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.547042    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0209970c-c0d2-4c96-87ef-9e27ffa077b6-crio-socket\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.547132 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.547064    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.547273 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:35.547155    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:35.547273 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.547168    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0209970c-c0d2-4c96-87ef-9e27ffa077b6-crio-socket\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.547273 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:35.547197    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:35.547273 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:35.547224    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:36.047206622 +0000 UTC m=+194.704916140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:35.547273 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:35.547248    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:36.047232487 +0000 UTC m=+194.704942009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:35.547434 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.547296    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0209970c-c0d2-4c96-87ef-9e27ffa077b6-data-volume\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:35.554976 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:35.554955    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrvc\" (UniqueName: \"kubernetes.io/projected/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-api-access-jvrvc\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:36.055174 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:36.055138    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:36.055337 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:36.055197    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:36.055337 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:36.055268    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:36.055337 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:36.055289    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:36.055337 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:36.055334    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:37.055318962 +0000 UTC m=+195.713028469 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:36.055546 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:36.055346    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:37.055339715 +0000 UTC m=+195.713049222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:36.511420 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:36.511397    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:29:36.511851 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:36.511688    2544 scope.go:117] "RemoveContainer" containerID="f0dc3b2cbbcc3565a27cbba8913aaadc3eb1cf1e6ed68da851b27b0475b23250"
Jul 27 19:29:36.511930 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:36.511858    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-dwtww_openshift-console-operator(bfbcdbee-4f5a-49ca-bb43-157da996dad9)\"" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" podUID="bfbcdbee-4f5a-49ca-bb43-157da996dad9"
Jul 27 19:29:37.063448 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:37.063411    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:37.063652 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:37.063478    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:37.063652 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:37.063558    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:37.063652 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:37.063576    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:37.063652 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:37.063641    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.063620366 +0000 UTC m=+197.721329878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:37.063851 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:37.063661    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:39.063652246 +0000 UTC m=+197.721361756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:37.074179 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:37.074161    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-757g8_088f9643-e34a-468f-b1be-432eaa0d2346/dns-node-resolver/0.log"
Jul 27 19:29:38.474535 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:38.474509    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xg9lr_78ac947a-ae79-4d31-a502-ec3efb1aafd6/node-ca/0.log"
Jul 27 19:29:39.083962 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.083926    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:39.084117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.084050    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:39.084117 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.084076    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:39.084117 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.084114    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:39.084238 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.084138    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:43.084121329 +0000 UTC m=+201.741830837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:39.084238 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.084153    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:43.084147153 +0000 UTC m=+201.741856660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:39.274655 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.274625    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nxgrc_07f8c00f-e1ad-4805-8ec3-3465493eddca/serve-healthcheck-canary/0.log"
Jul 27 19:29:39.689441 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689404    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689453    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689472    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689502    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689519    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689568    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689578    2544 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:39.689591    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689581    2544 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689654    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.689635278 +0000 UTC m=+214.347344808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : secret "router-metrics-certs-default" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689643    2544 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689666    2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689681    2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75ccb94d97-g2mbc: secret "image-registry-tls" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689681    2544 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689689    2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689705    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle podName:2e0d2ae6-7377-4a43-8430-3b65150f65af nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.689687043 +0000 UTC m=+214.347396553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle") pod "router-default-645bb56f9f-b8xdb" (UID: "2e0d2ae6-7377-4a43-8430-3b65150f65af") : configmap references non-existent config key: service-ca.crt
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689720    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls podName:64904d6b-c545-4604-88e5-b7a5d38a4d37 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.689713257 +0000 UTC m=+214.347422764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls") pod "cluster-samples-operator-6748489656-phbk2" (UID: "64904d6b-c545-4604-88e5-b7a5d38a4d37") : secret "samples-operator-tls" not found
Jul 27 19:29:39.689910 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689801    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert podName:9fd028b2-a23b-416a-92bc-2f7e15ae1ae7 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.68979435 +0000 UTC m=+214.347503857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-4hhz6" (UID: "9fd028b2-a23b-416a-92bc-2f7e15ae1ae7") : secret "networking-console-plugin-cert" not found
Jul 27 19:29:39.690394 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689813    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls podName:c6c6e6e0-40d7-46cc-8d0a-bbec77770677 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.689806906 +0000 UTC m=+214.347516413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls") pod "image-registry-75ccb94d97-g2mbc" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677") : secret "image-registry-tls" not found
Jul 27 19:29:39.690394 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689822    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls podName:2f225d20-1fb1-4cd0-bf76-31d0b51f791f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.689817035 +0000 UTC m=+214.347526542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls") pod "dns-default-zqrl2" (UID: "2f225d20-1fb1-4cd0-bf76-31d0b51f791f") : secret "dns-default-metrics-tls" not found
Jul 27 19:29:39.690394 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:39.689831    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls podName:4fceebef-ef4a-4f7d-af19-a773ba51379f nodeName:}" failed. No retries permitted until 2026-07-27 19:29:55.689826243 +0000 UTC m=+214.347535750 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6nhm9" (UID: "4fceebef-ef4a-4f7d-af19-a773ba51379f") : secret "cluster-monitoring-operator-tls" not found
Jul 27 19:29:43.122461 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:43.122420    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:43.122849 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:43.122492    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:43.122849 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:43.122558    2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 19:29:43.122849 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:43.122631    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:51.122612771 +0000 UTC m=+209.780322283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : configmap "kube-rbac-proxy" not found
Jul 27 19:29:43.122849 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:43.122658    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:43.122849 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:43.122715    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:29:51.122701262 +0000 UTC m=+209.780410769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:44.134429 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:44.134394    2544 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:44.134841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:44.134449    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:44.134841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:44.134800    2544 scope.go:117] "RemoveContainer" containerID="f0dc3b2cbbcc3565a27cbba8913aaadc3eb1cf1e6ed68da851b27b0475b23250"
Jul 27 19:29:44.134963 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:44.134945    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-dwtww_openshift-console-operator(bfbcdbee-4f5a-49ca-bb43-157da996dad9)\"" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" podUID="bfbcdbee-4f5a-49ca-bb43-157da996dad9"
Jul 27 19:29:51.191003 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:51.190969    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:51.191459 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:51.191022    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:51.191459 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:51.191165    2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:51.191459 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:51.191219    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls podName:0209970c-c0d2-4c96-87ef-9e27ffa077b6 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:07.191204269 +0000 UTC m=+225.848913777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-92r4h" (UID: "0209970c-c0d2-4c96-87ef-9e27ffa077b6") : secret "insights-runtime-extractor-tls" not found
Jul 27 19:29:51.191629 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:51.191606    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0209970c-c0d2-4c96-87ef-9e27ffa077b6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:29:55.729970 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.729936    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:55.729970 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.729970    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:55.730437 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.729999    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:55.730437 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.730029    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:55.730437 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.730066    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:55.730437 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.730089    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:55.730437 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.730158    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:55.730776 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.730726    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e0d2ae6-7377-4a43-8430-3b65150f65af-service-ca-bundle\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:55.732521 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.732489    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f225d20-1fb1-4cd0-bf76-31d0b51f791f-metrics-tls\") pod \"dns-default-zqrl2\" (UID: \"2f225d20-1fb1-4cd0-bf76-31d0b51f791f\") " pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:55.732629 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.732605    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e0d2ae6-7377-4a43-8430-3b65150f65af-metrics-certs\") pod \"router-default-645bb56f9f-b8xdb\" (UID: \"2e0d2ae6-7377-4a43-8430-3b65150f65af\") " pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:55.732931 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.732913    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/64904d6b-c545-4604-88e5-b7a5d38a4d37-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-phbk2\" (UID: \"64904d6b-c545-4604-88e5-b7a5d38a4d37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:55.732968 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.732938    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fceebef-ef4a-4f7d-af19-a773ba51379f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6nhm9\" (UID: \"4fceebef-ef4a-4f7d-af19-a773ba51379f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:55.733057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.733039    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fd028b2-a23b-416a-92bc-2f7e15ae1ae7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-4hhz6\" (UID: \"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:55.733290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.733275    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"image-registry-75ccb94d97-g2mbc\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") " pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:55.942656 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.942622    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"
Jul 27 19:29:55.965687 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.965646    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:55.973757 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.973727    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"
Jul 27 19:29:55.980837 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:55.980794    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"
Jul 27 19:29:56.004332 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.004306    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:56.009517 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.009491    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:29:56.162277 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.145759    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9"]
Jul 27 19:29:56.182470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.182379    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75ccb94d97-g2mbc"]
Jul 27 19:29:56.188218 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:56.188181    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c6e6e0_40d7_46cc_8d0a_bbec77770677.slice/crio-7c50bda91151e54e8a257168d29fd8a1898262c4ffc910449c63ab9b163f8733 WatchSource:0}: Error finding container 7c50bda91151e54e8a257168d29fd8a1898262c4ffc910449c63ab9b163f8733: Status 404 returned error can't find the container with id 7c50bda91151e54e8a257168d29fd8a1898262c4ffc910449c63ab9b163f8733
Jul 27 19:29:56.219670 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.219643    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zqrl2"]
Jul 27 19:29:56.224138 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:56.223953    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f225d20_1fb1_4cd0_bf76_31d0b51f791f.slice/crio-a1de05106d073e3622532e505f19077b74d6607a856bf9a8336a72648321c790 WatchSource:0}: Error finding container a1de05106d073e3622532e505f19077b74d6607a856bf9a8336a72648321c790: Status 404 returned error can't find the container with id a1de05106d073e3622532e505f19077b74d6607a856bf9a8336a72648321c790
Jul 27 19:29:56.259535 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.259470    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2"]
Jul 27 19:29:56.282758 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.282715    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6"]
Jul 27 19:29:56.285368 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:56.285342    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd028b2_a23b_416a_92bc_2f7e15ae1ae7.slice/crio-bcbc1865400bce9ada84593abe44eedc5722738fa43ea3b360c1b59f2455fb2c WatchSource:0}: Error finding container bcbc1865400bce9ada84593abe44eedc5722738fa43ea3b360c1b59f2455fb2c: Status 404 returned error can't find the container with id bcbc1865400bce9ada84593abe44eedc5722738fa43ea3b360c1b59f2455fb2c
Jul 27 19:29:56.302859 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.302837    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-645bb56f9f-b8xdb"]
Jul 27 19:29:56.306235 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:29:56.306211    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0d2ae6_7377_4a43_8430_3b65150f65af.slice/crio-835fed4774382eafc0d91da0ca091b6aa1d4baaef6129015b26a5ae0adeacd10 WatchSource:0}: Error finding container 835fed4774382eafc0d91da0ca091b6aa1d4baaef6129015b26a5ae0adeacd10: Status 404 returned error can't find the container with id 835fed4774382eafc0d91da0ca091b6aa1d4baaef6129015b26a5ae0adeacd10
Jul 27 19:29:56.577445 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.577364    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zqrl2" event={"ID":"2f225d20-1fb1-4cd0-bf76-31d0b51f791f","Type":"ContainerStarted","Data":"a1de05106d073e3622532e505f19077b74d6607a856bf9a8336a72648321c790"}
Jul 27 19:29:56.578895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.578850    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-645bb56f9f-b8xdb" event={"ID":"2e0d2ae6-7377-4a43-8430-3b65150f65af","Type":"ContainerStarted","Data":"a2f5aff71c0fa4fbad2b386a01ad2204f991da3f19a6bbd60410dc11a3e12229"}
Jul 27 19:29:56.579041 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.578908    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-645bb56f9f-b8xdb" event={"ID":"2e0d2ae6-7377-4a43-8430-3b65150f65af","Type":"ContainerStarted","Data":"835fed4774382eafc0d91da0ca091b6aa1d4baaef6129015b26a5ae0adeacd10"}
Jul 27 19:29:56.580079 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.580049    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6" event={"ID":"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7","Type":"ContainerStarted","Data":"bcbc1865400bce9ada84593abe44eedc5722738fa43ea3b360c1b59f2455fb2c"}
Jul 27 19:29:56.581185 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.581152    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2" event={"ID":"64904d6b-c545-4604-88e5-b7a5d38a4d37","Type":"ContainerStarted","Data":"e9e381f0e5f1955fce57dedd5078444e7cfabb3935e8a1d79a9376e3567835f2"}
Jul 27 19:29:56.582401 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.582373    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9" event={"ID":"4fceebef-ef4a-4f7d-af19-a773ba51379f","Type":"ContainerStarted","Data":"939b16ce7f7dc7afe44946d2874603bac1796e5bb4a881421150a731bdc593c0"}
Jul 27 19:29:56.583787 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.583764    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc" event={"ID":"c6c6e6e0-40d7-46cc-8d0a-bbec77770677","Type":"ContainerStarted","Data":"b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d"}
Jul 27 19:29:56.583927 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.583801    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc" event={"ID":"c6c6e6e0-40d7-46cc-8d0a-bbec77770677","Type":"ContainerStarted","Data":"7c50bda91151e54e8a257168d29fd8a1898262c4ffc910449c63ab9b163f8733"}
Jul 27 19:29:56.583927 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.583847    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:29:56.598552 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.598495    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-645bb56f9f-b8xdb" podStartSLOduration=148.598474202 podStartE2EDuration="2m28.598474202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:56.597414751 +0000 UTC m=+215.255124280" watchObservedRunningTime="2026-07-27 19:29:56.598474202 +0000 UTC m=+215.256183733"
Jul 27 19:29:56.615368 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.615326    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc" podStartSLOduration=148.615312751 podStartE2EDuration="2m28.615312751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:56.614236026 +0000 UTC m=+215.271945560" watchObservedRunningTime="2026-07-27 19:29:56.615312751 +0000 UTC m=+215.273022278"
Jul 27 19:29:56.835454 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:56.835389    2544 scope.go:117] "RemoveContainer" containerID="f0dc3b2cbbcc3565a27cbba8913aaadc3eb1cf1e6ed68da851b27b0475b23250"
Jul 27 19:29:57.004923 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.004650    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:57.008399 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.008160    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:57.601081 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.601052    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:29:57.603032 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.601938    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" event={"ID":"bfbcdbee-4f5a-49ca-bb43-157da996dad9","Type":"ContainerStarted","Data":"a0366efbda804fbc43fea2dd47a894b1e73e5e5e09c6220b8253f4e3dc42d488"}
Jul 27 19:29:57.603032 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.602001    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:57.603032 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.602983    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:57.603885 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.603827    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-645bb56f9f-b8xdb"
Jul 27 19:29:57.642190 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.641090    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww" podStartSLOduration=140.032159442 podStartE2EDuration="2m29.641069105s" totalImagesPullingTime="9.608909663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:29:57.621879244 +0000 UTC m=+216.279588776" watchObservedRunningTime="2026-07-27 19:29:57.641069105 +0000 UTC m=+216.298778635"
Jul 27 19:29:57.922275 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:57.922229    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-dwtww"
Jul 27 19:29:58.481206 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:29:58.481178    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75ccb94d97-g2mbc"]
Jul 27 19:29:59.847520 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:29:59.847482    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7d4b9" podUID="f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01"
Jul 27 19:30:00.613354 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.613319    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6" event={"ID":"9fd028b2-a23b-416a-92bc-2f7e15ae1ae7","Type":"ContainerStarted","Data":"9a00da4638e72bd2d3d2ee6f69d51c2758565194952e78420e019bef155b0e17"}
Jul 27 19:30:00.615330 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.615298    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2" event={"ID":"64904d6b-c545-4604-88e5-b7a5d38a4d37","Type":"ContainerStarted","Data":"d944c3fe577801c3cf561ef4cab2dfdd76886a7aff94dd5b6a6df465c1e344f7"}
Jul 27 19:30:00.615483 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.615360    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2" event={"ID":"64904d6b-c545-4604-88e5-b7a5d38a4d37","Type":"ContainerStarted","Data":"6ac8935fe5767d74788f107c8dd46cd27d69ddf25f9b87f2a85641d2338138d2"}
Jul 27 19:30:00.617043 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.617012    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9" event={"ID":"4fceebef-ef4a-4f7d-af19-a773ba51379f","Type":"ContainerStarted","Data":"7735a522028aad0cb3b4599562de84ceec79a0a6aa388778ecd5665bd99ec951"}
Jul 27 19:30:00.618931 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.618906    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zqrl2" event={"ID":"2f225d20-1fb1-4cd0-bf76-31d0b51f791f","Type":"ContainerStarted","Data":"0388fde1b8a09346a1b991f1f4b4254d3c17c22a5e86e86dada6869f37a9befc"}
Jul 27 19:30:00.619062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.618951    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zqrl2" event={"ID":"2f225d20-1fb1-4cd0-bf76-31d0b51f791f","Type":"ContainerStarted","Data":"e76242e3f6d6669b48f8d1ee84791acc3472b268847cf521be49556c0793987d"}
Jul 27 19:30:00.619062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.619023    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:30:00.642387 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.642330    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-4hhz6" podStartSLOduration=148.70878719 podStartE2EDuration="2m32.64231171s" totalImagesPullingTime="3.93352452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:00.639872097 +0000 UTC m=+219.297581622" watchObservedRunningTime="2026-07-27 19:30:00.64231171 +0000 UTC m=+219.300021240"
Jul 27 19:30:00.674919 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.674862    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-phbk2" podStartSLOduration=148.760854047 podStartE2EDuration="2m32.67484234s" totalImagesPullingTime="3.913988293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:00.674105442 +0000 UTC m=+219.331814974" watchObservedRunningTime="2026-07-27 19:30:00.67484234 +0000 UTC m=+219.332551869"
Jul 27 19:30:00.700029 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.699985    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zqrl2" podStartSLOduration=33.707678651 podStartE2EDuration="37.699970498s" totalImagesPullingTime="3.992291847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:00.699071976 +0000 UTC m=+219.356781542" watchObservedRunningTime="2026-07-27 19:30:00.699970498 +0000 UTC m=+219.357680027"
Jul 27 19:30:00.716970 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:00.716923    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6nhm9" podStartSLOduration=148.657231445 podStartE2EDuration="2m32.716905378s" totalImagesPullingTime="4.059673933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:00.714876558 +0000 UTC m=+219.372586088" watchObservedRunningTime="2026-07-27 19:30:00.716905378 +0000 UTC m=+219.374614906"
Jul 27 19:30:07.228010 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:07.227972    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:30:07.230153 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:07.230119    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0209970c-c0d2-4c96-87ef-9e27ffa077b6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-92r4h\" (UID: \"0209970c-c0d2-4c96-87ef-9e27ffa077b6\") " pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:30:07.407356 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:07.407324    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-92r4h"
Jul 27 19:30:07.531671 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:07.531511    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-92r4h"]
Jul 27 19:30:07.534227 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:30:07.534205    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0209970c_c0d2_4c96_87ef_9e27ffa077b6.slice/crio-1da6c024b599295fee44266f768ee65227bb530890bccc50dfac12cf9a5f14d5 WatchSource:0}: Error finding container 1da6c024b599295fee44266f768ee65227bb530890bccc50dfac12cf9a5f14d5: Status 404 returned error can't find the container with id 1da6c024b599295fee44266f768ee65227bb530890bccc50dfac12cf9a5f14d5
Jul 27 19:30:07.644525 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:07.644492    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92r4h" event={"ID":"0209970c-c0d2-4c96-87ef-9e27ffa077b6","Type":"ContainerStarted","Data":"1da6c024b599295fee44266f768ee65227bb530890bccc50dfac12cf9a5f14d5"}
Jul 27 19:30:10.214788 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.214711    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-46qdz"]
Jul 27 19:30:10.221700 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.221673    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.224488 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.224462    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 19:30:10.225047 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.224834    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 19:30:10.225047 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.224843    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 19:30:10.225047 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.224933    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 19:30:10.225587 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.225452    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j8v9r\""
Jul 27 19:30:10.355219 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355180    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrbb2\" (UniqueName: \"kubernetes.io/projected/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-kube-api-access-wrbb2\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355409 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355309    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-textfile\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355409 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355343    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-accelerators-collector-config\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355409 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355379    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-root\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355409 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355402    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-wtmp\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355481    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-tls\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355520    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355547    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-metrics-client-ca\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.355582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.355573    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-sys\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456536 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456504    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-root\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456699 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456545    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-wtmp\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456699 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456582    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-tls\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456699 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456604    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456699 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456632    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-metrics-client-ca\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456699 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456645    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-root\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.456699 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456661    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-sys\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456746    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-wtmp\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456758    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrbb2\" (UniqueName: \"kubernetes.io/projected/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-kube-api-access-wrbb2\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:30:10.456787    2544 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456784    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-sys\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:30:10.456853    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-tls podName:54baa635-9a8c-43a0-bd9e-6c1cbb63b156 nodeName:}" failed. No retries permitted until 2026-07-27 19:30:10.956826077 +0000 UTC m=+229.614535593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-tls") pod "node-exporter-46qdz" (UID: "54baa635-9a8c-43a0-bd9e-6c1cbb63b156") : secret "node-exporter-tls" not found
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456893    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-textfile\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.456923    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-accelerators-collector-config\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457406 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.457145    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-textfile\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457406 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.457243    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-metrics-client-ca\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.457494 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.457427    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-accelerators-collector-config\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.459158 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.459137    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.467226 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.467172    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrbb2\" (UniqueName: \"kubernetes.io/projected/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-kube-api-access-wrbb2\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.625383 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.625350    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zqrl2"
Jul 27 19:30:10.961082 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.961044    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-tls\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:10.963531 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:10.963459    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54baa635-9a8c-43a0-bd9e-6c1cbb63b156-node-exporter-tls\") pod \"node-exporter-46qdz\" (UID: \"54baa635-9a8c-43a0-bd9e-6c1cbb63b156\") " pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:11.134193 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:11.134162    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-46qdz"
Jul 27 19:30:12.342407 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:30:12.342365    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54baa635_9a8c_43a0_bd9e_6c1cbb63b156.slice/crio-593db97e63793cdc5363ba9e50c1f4d4b6b020a373d12d0e01f18b042c9bf3b7 WatchSource:0}: Error finding container 593db97e63793cdc5363ba9e50c1f4d4b6b020a373d12d0e01f18b042c9bf3b7: Status 404 returned error can't find the container with id 593db97e63793cdc5363ba9e50c1f4d4b6b020a373d12d0e01f18b042c9bf3b7
Jul 27 19:30:12.666932 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:12.666902    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92r4h" event={"ID":"0209970c-c0d2-4c96-87ef-9e27ffa077b6","Type":"ContainerStarted","Data":"73077dd51f55d54c16e627f5313823bb97a263f623f20f680ff3fef3b6a681b8"}
Jul 27 19:30:12.668137 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:12.668109    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46qdz" event={"ID":"54baa635-9a8c-43a0-bd9e-6c1cbb63b156","Type":"ContainerStarted","Data":"593db97e63793cdc5363ba9e50c1f4d4b6b020a373d12d0e01f18b042c9bf3b7"}
Jul 27 19:30:12.835025 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:12.834993    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:30:13.677153 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:13.677080    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46qdz" event={"ID":"54baa635-9a8c-43a0-bd9e-6c1cbb63b156","Type":"ContainerStarted","Data":"d86759a6c249303cf48efcc9801776bd24e56f4e132a68ea316fbb3465e0cc33"}
Jul 27 19:30:14.684274 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:14.684230    2544 generic.go:356] "Generic (PLEG): container finished" podID="54baa635-9a8c-43a0-bd9e-6c1cbb63b156" containerID="d86759a6c249303cf48efcc9801776bd24e56f4e132a68ea316fbb3465e0cc33" exitCode=0
Jul 27 19:30:14.684754 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:14.684290    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46qdz" event={"ID":"54baa635-9a8c-43a0-bd9e-6c1cbb63b156","Type":"ContainerDied","Data":"d86759a6c249303cf48efcc9801776bd24e56f4e132a68ea316fbb3465e0cc33"}
Jul 27 19:30:16.694707 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:16.694663    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92r4h" event={"ID":"0209970c-c0d2-4c96-87ef-9e27ffa077b6","Type":"ContainerStarted","Data":"6875de9d3349b3d9d8a644bb544e4b3152be0b0355bc22282ffbcfe27e525fc4"}
Jul 27 19:30:16.697117 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:16.697087    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46qdz" event={"ID":"54baa635-9a8c-43a0-bd9e-6c1cbb63b156","Type":"ContainerStarted","Data":"abd026037d948050c28b474c70271998f4ea9efc152d85d901e28a3c257242c4"}
Jul 27 19:30:16.697261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:16.697133    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-46qdz" event={"ID":"54baa635-9a8c-43a0-bd9e-6c1cbb63b156","Type":"ContainerStarted","Data":"6f501584fc6d888797253e9e12439a4a70a24cb5e4a84a0199be30e4701df5b8"}
Jul 27 19:30:16.717244 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:16.717191    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-46qdz" podStartSLOduration=5.426977496 podStartE2EDuration="6.717177346s" totalImagesPullingTime="1.213451393s" totalInitContainerRuntime="76.748457ms" isStatefulPod=true podCreationTimestamp="2026-07-27 19:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:16.715432713 +0000 UTC m=+235.373142243" watchObservedRunningTime="2026-07-27 19:30:16.717177346 +0000 UTC m=+235.374886875"
Jul 27 19:30:18.611387 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:18.611358    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:30:19.710338 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:19.710296    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-92r4h" event={"ID":"0209970c-c0d2-4c96-87ef-9e27ffa077b6","Type":"ContainerStarted","Data":"bb64d7943ee7895230df74cf52cd0071b3fae125a6f992b98433d5a14318813d"}
Jul 27 19:30:19.728273 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:19.728223    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-92r4h" podStartSLOduration=33.127700225 podStartE2EDuration="44.728208014s" totalImagesPullingTime="11.600507789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 19:29:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:30:19.726882524 +0000 UTC m=+238.384592074" watchObservedRunningTime="2026-07-27 19:30:19.728208014 +0000 UTC m=+238.385917542"
Jul 27 19:30:23.623042 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.622991    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc" podUID="c6c6e6e0-40d7-46cc-8d0a-bbec77770677" containerName="registry" containerID="cri-o://b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d" gracePeriod=30
Jul 27 19:30:23.862011 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.861988    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:30:23.976007 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.975923    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlntq\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-kube-api-access-jlntq\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976148 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976014    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-installation-pull-secrets\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976148 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976038    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-bound-sa-token\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976148 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976140    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-trusted-ca\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976307 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976163    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-certificates\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976307 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976187    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-ca-trust-extracted\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976307 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976212    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976307 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976251    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-image-registry-private-configuration\") pod \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\" (UID: \"c6c6e6e0-40d7-46cc-8d0a-bbec77770677\") "
Jul 27 19:30:23.976895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976594    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:23.976895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.976614    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:30:23.978519 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.978486    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:23.978635 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.978535    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-kube-api-access-jlntq" (OuterVolumeSpecName: "kube-api-access-jlntq") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "kube-api-access-jlntq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:23.978635 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.978545    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:23.978865 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.978843    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:30:23.978924 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.978861    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:30:23.984916 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:23.984889    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c6c6e6e0-40d7-46cc-8d0a-bbec77770677" (UID: "c6c6e6e0-40d7-46cc-8d0a-bbec77770677"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:30:24.077789 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077759    2544 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-installation-pull-secrets\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077789 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077787    2544 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-bound-sa-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077796    2544 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-trusted-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077816    2544 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-certificates\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077825    2544 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-ca-trust-extracted\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077833    2544 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-registry-tls\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077842    2544 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-image-registry-private-configuration\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.077948 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.077850    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlntq\" (UniqueName: \"kubernetes.io/projected/c6c6e6e0-40d7-46cc-8d0a-bbec77770677-kube-api-access-jlntq\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:30:24.728200 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.728167    2544 generic.go:356] "Generic (PLEG): container finished" podID="c6c6e6e0-40d7-46cc-8d0a-bbec77770677" containerID="b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d" exitCode=0
Jul 27 19:30:24.728686 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.728214    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc"
Jul 27 19:30:24.728686 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.728255    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc" event={"ID":"c6c6e6e0-40d7-46cc-8d0a-bbec77770677","Type":"ContainerDied","Data":"b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d"}
Jul 27 19:30:24.728686 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.728307    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75ccb94d97-g2mbc" event={"ID":"c6c6e6e0-40d7-46cc-8d0a-bbec77770677","Type":"ContainerDied","Data":"7c50bda91151e54e8a257168d29fd8a1898262c4ffc910449c63ab9b163f8733"}
Jul 27 19:30:24.728686 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.728328    2544 scope.go:117] "RemoveContainer" containerID="b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d"
Jul 27 19:30:24.736652 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.736635    2544 scope.go:117] "RemoveContainer" containerID="b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d"
Jul 27 19:30:24.736935 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:30:24.736918    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d\": container with ID starting with b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d not found: ID does not exist" containerID="b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d"
Jul 27 19:30:24.736998 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.736943    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d"} err="failed to get container status \"b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d\": rpc error: code = NotFound desc = could not find container \"b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d\": container with ID starting with b8eb5e864d42a49ac0749235974e4b2cc1cfe9d7ff63f0c312105cf71a67aa9d not found: ID does not exist"
Jul 27 19:30:24.752149 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.750325    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75ccb94d97-g2mbc"]
Jul 27 19:30:24.753007 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:24.752983    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75ccb94d97-g2mbc"]
Jul 27 19:30:25.839008 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:25.838970    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6c6e6e0-40d7-46cc-8d0a-bbec77770677" path="/var/lib/kubelet/pods/c6c6e6e0-40d7-46cc-8d0a-bbec77770677/volumes"
Jul 27 19:30:39.783062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:39.783034    2544 generic.go:356] "Generic (PLEG): container finished" podID="c6b55b08-2fc0-4416-bba2-bd4f0d5f6308" containerID="ee861a2c74f0a962aacb7617d63ab6a845031417fb37f378e73d3866f5cda582" exitCode=0
Jul 27 19:30:39.783360 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:39.783109    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-62tqr" event={"ID":"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308","Type":"ContainerDied","Data":"ee861a2c74f0a962aacb7617d63ab6a845031417fb37f378e73d3866f5cda582"}
Jul 27 19:30:39.783401 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:39.783391    2544 scope.go:117] "RemoveContainer" containerID="ee861a2c74f0a962aacb7617d63ab6a845031417fb37f378e73d3866f5cda582"
Jul 27 19:30:40.788483 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:40.788452    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-62tqr" event={"ID":"c6b55b08-2fc0-4416-bba2-bd4f0d5f6308","Type":"ContainerStarted","Data":"01298a47b3fb8ebe4b537cd5bed7f4f9d7330059d028a36436c04674ac8081ce"}
Jul 27 19:30:41.363266 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:41.363233    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-6nhm9_4fceebef-ef4a-4f7d-af19-a773ba51379f/cluster-monitoring-operator/0.log"
Jul 27 19:30:42.561833 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:42.561775    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46qdz_54baa635-9a8c-43a0-bd9e-6c1cbb63b156/init-textfile/0.log"
Jul 27 19:30:42.759814 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:42.759789    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46qdz_54baa635-9a8c-43a0-bd9e-6c1cbb63b156/node-exporter/0.log"
Jul 27 19:30:42.958233 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:42.958205    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46qdz_54baa635-9a8c-43a0-bd9e-6c1cbb63b156/kube-rbac-proxy/0.log"
Jul 27 19:30:48.770566 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:48.770486    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4hhz6_9fd028b2-a23b-416a-92bc-2f7e15ae1ae7/networking-console-plugin/0.log"
Jul 27 19:30:48.958407 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:48.958382    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:30:49.161365 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:30:49.161333    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/2.log"
Jul 27 19:31:00.855969 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:00.855937    2544 generic.go:356] "Generic (PLEG): container finished" podID="2fdf52d3-779d-440d-aace-29b908c7e658" containerID="00d9ba69ced9cba0d802dc7afae87da96ed2dce267abb9dd3818fbc969437f39" exitCode=0
Jul 27 19:31:00.856344 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:00.856009    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl" event={"ID":"2fdf52d3-779d-440d-aace-29b908c7e658","Type":"ContainerDied","Data":"00d9ba69ced9cba0d802dc7afae87da96ed2dce267abb9dd3818fbc969437f39"}
Jul 27 19:31:00.856344 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:00.856303    2544 scope.go:117] "RemoveContainer" containerID="00d9ba69ced9cba0d802dc7afae87da96ed2dce267abb9dd3818fbc969437f39"
Jul 27 19:31:01.860559 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:01.860530    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-5qljl" event={"ID":"2fdf52d3-779d-440d-aace-29b908c7e658","Type":"ContainerStarted","Data":"ae3ef2a8c5f47a25d1548e8558016c5e4481bb59099483550c8154a7822415a7"}
Jul 27 19:31:04.251486 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:04.251450    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" podUID="720c7565-d143-44b1-9fae-35306effed02" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 19:31:05.874103 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:05.874073    2544 generic.go:356] "Generic (PLEG): container finished" podID="0af76269-e664-48e9-addb-350ebc791abd" containerID="20c889bae0e9f458b87e5868f08e0d12d1251e245c186e4b8fffb7faddc0dad2" exitCode=0
Jul 27 19:31:05.874441 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:05.874143    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5" event={"ID":"0af76269-e664-48e9-addb-350ebc791abd","Type":"ContainerDied","Data":"20c889bae0e9f458b87e5868f08e0d12d1251e245c186e4b8fffb7faddc0dad2"}
Jul 27 19:31:05.874441 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:05.874390    2544 scope.go:117] "RemoveContainer" containerID="20c889bae0e9f458b87e5868f08e0d12d1251e245c186e4b8fffb7faddc0dad2"
Jul 27 19:31:06.879640 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:06.879605    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5" event={"ID":"0af76269-e664-48e9-addb-350ebc791abd","Type":"ContainerStarted","Data":"a035b3913ac9c09aebaea254def157ae17a10031440dc85563565774d8ac01cd"}
Jul 27 19:31:14.251285 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:14.251248    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" podUID="720c7565-d143-44b1-9fae-35306effed02" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 19:31:21.694937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:21.694910    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:31:21.698836 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:21.698811    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:31:21.704996 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:21.704976    2544 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 19:31:24.251232 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.251199    2544 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" podUID="720c7565-d143-44b1-9fae-35306effed02" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 19:31:24.252799 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.251255    2544 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6"
Jul 27 19:31:24.252799 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.251659    2544 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"77671cc898a2b202d61514a849b5e875c48308a6a4386c7d28424a3a75a5d892"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 27 19:31:24.252799 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.251694    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" podUID="720c7565-d143-44b1-9fae-35306effed02" containerName="service-proxy" containerID="cri-o://77671cc898a2b202d61514a849b5e875c48308a6a4386c7d28424a3a75a5d892" gracePeriod=30
Jul 27 19:31:24.942828 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.942798    2544 generic.go:356] "Generic (PLEG): container finished" podID="720c7565-d143-44b1-9fae-35306effed02" containerID="77671cc898a2b202d61514a849b5e875c48308a6a4386c7d28424a3a75a5d892" exitCode=2
Jul 27 19:31:24.942974 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.942834    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" event={"ID":"720c7565-d143-44b1-9fae-35306effed02","Type":"ContainerDied","Data":"77671cc898a2b202d61514a849b5e875c48308a6a4386c7d28424a3a75a5d892"}
Jul 27 19:31:24.942974 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:24.942865    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bc9f4ff98-59bt6" event={"ID":"720c7565-d143-44b1-9fae-35306effed02","Type":"ContainerStarted","Data":"7aeda038d30e36dee3c5e437a6c4e8c8c42305b4782d9ec16cd3b272accdc9d8"}
Jul 27 19:31:33.675276 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.675230    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:31:33.677440 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.677421    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01-metrics-certs\") pod \"network-metrics-daemon-7d4b9\" (UID: \"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01\") " pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:31:33.838363 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.838332    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lj4rj\""
Jul 27 19:31:33.846404 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.846382    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d4b9"
Jul 27 19:31:33.966864 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.966677    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7d4b9"]
Jul 27 19:31:33.969449 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:31:33.969424    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a7a022_6cc9_4aca_8c4d_8e18d93d1a01.slice/crio-a9036ec6b6172060f30348c9ad76c0a50296537d1c108c58cf06ad39ded855bc WatchSource:0}: Error finding container a9036ec6b6172060f30348c9ad76c0a50296537d1c108c58cf06ad39ded855bc: Status 404 returned error can't find the container with id a9036ec6b6172060f30348c9ad76c0a50296537d1c108c58cf06ad39ded855bc
Jul 27 19:31:33.971415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.971392    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:31:33.975273 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:33.975246    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7d4b9" event={"ID":"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01","Type":"ContainerStarted","Data":"a9036ec6b6172060f30348c9ad76c0a50296537d1c108c58cf06ad39ded855bc"}
Jul 27 19:31:35.985394 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:35.985349    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7d4b9" event={"ID":"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01","Type":"ContainerStarted","Data":"b531121cc3381276300f08e6ca8ca76d4a84744075647c3a06ab1318dbfc3c2a"}
Jul 27 19:31:35.985779 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:35.985415    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7d4b9" event={"ID":"f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01","Type":"ContainerStarted","Data":"87e27ebed339bd908df4f22015959b7c8636d5795da772498e8d3c7ef2ed3a4f"}
Jul 27 19:31:36.002363 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:31:36.002304    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7d4b9" podStartSLOduration=258.947450124 podStartE2EDuration="4m20.002290283s" totalImagesPullingTime="1.054840159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:31:36.001250609 +0000 UTC m=+314.658960140" watchObservedRunningTime="2026-07-27 19:31:36.002290283 +0000 UTC m=+314.659999811"
Jul 27 19:33:37.206240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.206203    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"]
Jul 27 19:33:37.206676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.206481    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6c6e6e0-40d7-46cc-8d0a-bbec77770677" containerName="registry"
Jul 27 19:33:37.206676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.206493    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6c6e6e0-40d7-46cc-8d0a-bbec77770677" containerName="registry"
Jul 27 19:33:37.206676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.206566    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6c6e6e0-40d7-46cc-8d0a-bbec77770677" containerName="registry"
Jul 27 19:33:37.208457 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.208440    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.211401 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.211377    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 19:33:37.211524 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.211475    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 19:33:37.211648 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.211631    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-grzqz\""
Jul 27 19:33:37.211715 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.211698    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 19:33:37.211774 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.211697    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 19:33:37.224869 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.224848    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"]
Jul 27 19:33:37.357994 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.357966    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-tmp\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.358169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.358000    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.358169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.358027    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-manifests\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.358169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.358103    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.358169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.358153    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdtx8\" (UniqueName: \"kubernetes.io/projected/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-kube-api-access-tdtx8\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.459011 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.458926    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdtx8\" (UniqueName: \"kubernetes.io/projected/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-kube-api-access-tdtx8\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.459011 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.458974    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-tmp\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.459011 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.458993    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.459287 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.459023    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-manifests\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.459287 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.459065    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.459486 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.459456    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-manifests\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.461321 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.461303    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-tmp\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.461516 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.461497    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.461560 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.461497    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.469154 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.469131    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdtx8\" (UniqueName: \"kubernetes.io/projected/b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7-kube-api-access-tdtx8\") pod \"opendatahub-operator-controller-manager-6c9f4fc9b4-486qn\" (UID: \"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7\") " pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.519147 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.519118    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:37.676796 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:37.676765    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"]
Jul 27 19:33:37.680219 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:33:37.680193    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ad9ecc_5d9f_48e7_a09f_0ecb66b4a0d7.slice/crio-3a6ac4b542b3deba739b3f4e220fb6f706129f43f56fe3589b8de38b4266c6d1 WatchSource:0}: Error finding container 3a6ac4b542b3deba739b3f4e220fb6f706129f43f56fe3589b8de38b4266c6d1: Status 404 returned error can't find the container with id 3a6ac4b542b3deba739b3f4e220fb6f706129f43f56fe3589b8de38b4266c6d1
Jul 27 19:33:38.433295 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:38.433244    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn" event={"ID":"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7","Type":"ContainerStarted","Data":"3a6ac4b542b3deba739b3f4e220fb6f706129f43f56fe3589b8de38b4266c6d1"}
Jul 27 19:33:40.443456 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:40.443417    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn" event={"ID":"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7","Type":"ContainerStarted","Data":"68b53116c2e165a4b81f6b28c1351bd666701155713e33e43fde23e35cdc82b9"}
Jul 27 19:33:41.449514 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:41.449485    2544 generic.go:356] "Generic (PLEG): container finished" podID="b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7" containerID="68b53116c2e165a4b81f6b28c1351bd666701155713e33e43fde23e35cdc82b9" exitCode=0
Jul 27 19:33:41.449903 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:41.449573    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn" event={"ID":"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7","Type":"ContainerDied","Data":"68b53116c2e165a4b81f6b28c1351bd666701155713e33e43fde23e35cdc82b9"}
Jul 27 19:33:42.455696 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:42.455662    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn" event={"ID":"b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7","Type":"ContainerStarted","Data":"0a5ade7d27bfc327e28c4d008692060e65e962846e698eaac0c96c0782afefa5"}
Jul 27 19:33:42.456089 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:42.455883    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:53.462631 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:53.462600    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn"
Jul 27 19:33:53.522697 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:53.522635    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c9f4fc9b4-486qn" podStartSLOduration=12.323198297 podStartE2EDuration="16.522618186s" totalImagesPullingTime="3.149795388s" totalInitContainerRuntime="1.049624501s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:33:42.482285167 +0000 UTC m=+441.139994695" watchObservedRunningTime="2026-07-27 19:33:53.522618186 +0000 UTC m=+452.180327714"
Jul 27 19:33:59.337971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.337938    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"]
Jul 27 19:33:59.348708 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.348682    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.355130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.355035    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 19:33:59.355130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.355106    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 19:33:59.355526 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.355509    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-z6zwf\""
Jul 27 19:33:59.373892 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.373870    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"]
Jul 27 19:33:59.440676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.440644    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqswq\" (UniqueName: \"kubernetes.io/projected/f61a9a12-e963-4935-8e03-b67c0b35f921-kube-api-access-dqswq\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.440863 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.440686    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61a9a12-e963-4935-8e03-b67c0b35f921-tls-certs\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.440863 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.440809    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f61a9a12-e963-4935-8e03-b67c0b35f921-tmp\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.521133 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.521104    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ntf9s"]
Jul 27 19:33:59.526697 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.526674    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.529461 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.529440    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2lgcz\""
Jul 27 19:33:59.529590 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.529465    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 27 19:33:59.539285 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.539261    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ntf9s"]
Jul 27 19:33:59.542136 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.542108    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgzmg\" (UniqueName: \"kubernetes.io/projected/693be74d-32c6-4fa1-ac4b-ce702bfc3910-kube-api-access-xgzmg\") pod \"odh-model-controller-7998c67dc4-ntf9s\" (UID: \"693be74d-32c6-4fa1-ac4b-ce702bfc3910\") " pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.542257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.542155    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/693be74d-32c6-4fa1-ac4b-ce702bfc3910-cert\") pod \"odh-model-controller-7998c67dc4-ntf9s\" (UID: \"693be74d-32c6-4fa1-ac4b-ce702bfc3910\") " pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.542257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.542185    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f61a9a12-e963-4935-8e03-b67c0b35f921-tmp\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.542352 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.542298    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqswq\" (UniqueName: \"kubernetes.io/projected/f61a9a12-e963-4935-8e03-b67c0b35f921-kube-api-access-dqswq\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.542352 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.542337    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61a9a12-e963-4935-8e03-b67c0b35f921-tls-certs\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.544562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.544531    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f61a9a12-e963-4935-8e03-b67c0b35f921-tmp\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.544845 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.544829    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61a9a12-e963-4935-8e03-b67c0b35f921-tls-certs\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.565348 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.565324    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqswq\" (UniqueName: \"kubernetes.io/projected/f61a9a12-e963-4935-8e03-b67c0b35f921-kube-api-access-dqswq\") pod \"kube-auth-proxy-5b7cdcf968-mgjtv\" (UID: \"f61a9a12-e963-4935-8e03-b67c0b35f921\") " pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.643387 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.643350    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/693be74d-32c6-4fa1-ac4b-ce702bfc3910-cert\") pod \"odh-model-controller-7998c67dc4-ntf9s\" (UID: \"693be74d-32c6-4fa1-ac4b-ce702bfc3910\") " pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.643600 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.643453    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgzmg\" (UniqueName: \"kubernetes.io/projected/693be74d-32c6-4fa1-ac4b-ce702bfc3910-kube-api-access-xgzmg\") pod \"odh-model-controller-7998c67dc4-ntf9s\" (UID: \"693be74d-32c6-4fa1-ac4b-ce702bfc3910\") " pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.645927 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.645893    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/693be74d-32c6-4fa1-ac4b-ce702bfc3910-cert\") pod \"odh-model-controller-7998c67dc4-ntf9s\" (UID: \"693be74d-32c6-4fa1-ac4b-ce702bfc3910\") " pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.652516 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.652482    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgzmg\" (UniqueName: \"kubernetes.io/projected/693be74d-32c6-4fa1-ac4b-ce702bfc3910-kube-api-access-xgzmg\") pod \"odh-model-controller-7998c67dc4-ntf9s\" (UID: \"693be74d-32c6-4fa1-ac4b-ce702bfc3910\") " pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:33:59.658162 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.658130    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"
Jul 27 19:33:59.796962 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.796930    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv"]
Jul 27 19:33:59.800337 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:33:59.800290    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61a9a12_e963_4935_8e03_b67c0b35f921.slice/crio-b585fd40dcec19d026458c059bea77caa6d823244ad38abe71d447a8ab2db249 WatchSource:0}: Error finding container b585fd40dcec19d026458c059bea77caa6d823244ad38abe71d447a8ab2db249: Status 404 returned error can't find the container with id b585fd40dcec19d026458c059bea77caa6d823244ad38abe71d447a8ab2db249
Jul 27 19:33:59.837573 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:33:59.837535    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:34:00.013426 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.013179    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ntf9s"]
Jul 27 19:34:00.015702 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:00.015674    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693be74d_32c6_4fa1_ac4b_ce702bfc3910.slice/crio-04400ef33af52649b8c9da4b16b9cd7a9948469c2d7b12436ffdb7cfadce9057 WatchSource:0}: Error finding container 04400ef33af52649b8c9da4b16b9cd7a9948469c2d7b12436ffdb7cfadce9057: Status 404 returned error can't find the container with id 04400ef33af52649b8c9da4b16b9cd7a9948469c2d7b12436ffdb7cfadce9057
Jul 27 19:34:00.260995 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.260919    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-klgvp"]
Jul 27 19:34:00.263824 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.263804    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.266311 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.266288    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 27 19:34:00.266442 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.266367    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8h6zr\""
Jul 27 19:34:00.275805 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.275778    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-klgvp"]
Jul 27 19:34:00.351112 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.351078    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28bc3dd5-b6ea-46ef-8142-d202775c9036-tls-certs\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.351112 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.351120    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scgcw\" (UniqueName: \"kubernetes.io/projected/28bc3dd5-b6ea-46ef-8142-d202775c9036-kube-api-access-scgcw\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.452257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.452220    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28bc3dd5-b6ea-46ef-8142-d202775c9036-tls-certs\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.452257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.452261    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scgcw\" (UniqueName: \"kubernetes.io/projected/28bc3dd5-b6ea-46ef-8142-d202775c9036-kube-api-access-scgcw\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.452493 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:00.452406    2544 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 27 19:34:00.452493 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:00.452481    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28bc3dd5-b6ea-46ef-8142-d202775c9036-tls-certs podName:28bc3dd5-b6ea-46ef-8142-d202775c9036 nodeName:}" failed. No retries permitted until 2026-07-27 19:34:00.952460678 +0000 UTC m=+459.610170194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/28bc3dd5-b6ea-46ef-8142-d202775c9036-tls-certs") pod "model-serving-api-54d89bd5fc-klgvp" (UID: "28bc3dd5-b6ea-46ef-8142-d202775c9036") : secret "model-serving-api-tls" not found
Jul 27 19:34:00.468340 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.468290    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scgcw\" (UniqueName: \"kubernetes.io/projected/28bc3dd5-b6ea-46ef-8142-d202775c9036-kube-api-access-scgcw\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.534599 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.534507    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s" event={"ID":"693be74d-32c6-4fa1-ac4b-ce702bfc3910","Type":"ContainerStarted","Data":"04400ef33af52649b8c9da4b16b9cd7a9948469c2d7b12436ffdb7cfadce9057"}
Jul 27 19:34:00.536192 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.536132    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv" event={"ID":"f61a9a12-e963-4935-8e03-b67c0b35f921","Type":"ContainerStarted","Data":"b585fd40dcec19d026458c059bea77caa6d823244ad38abe71d447a8ab2db249"}
Jul 27 19:34:00.666619 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.666189    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"]
Jul 27 19:34:00.669019 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.668996    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.684261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.683286    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 27 19:34:00.684261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.683547    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ln5c8\""
Jul 27 19:34:00.684261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.683725    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 27 19:34:00.684261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.683904    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 19:34:00.684261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.684121    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 27 19:34:00.694067 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.693998    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"]
Jul 27 19:34:00.755200 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755157    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.755377 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755220    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.755377 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755282    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.755377 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755330    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.755377 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755360    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjfwc\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-kube-api-access-rjfwc\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.755594 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755407    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65e628d-169b-4c41-85b3-23b3abb338c0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.755594 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.755429    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.856877 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.856790    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.856877 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.856869    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.857094 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.856911    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.857094 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.856998    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.857094 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.857049    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.857094 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.857084    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjfwc\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-kube-api-access-rjfwc\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.857286 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.857133    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65e628d-169b-4c41-85b3-23b3abb338c0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.858168 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.857913    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.861062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.861033    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65e628d-169b-4c41-85b3-23b3abb338c0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.861225 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.861203    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.861773 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.861713    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.862190 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.862166    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.876017 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.875968    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.877798 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.877754    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjfwc\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-kube-api-access-rjfwc\") pod \"istiod-openshift-gateway-7b7995d5cd-clh22\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:00.958808 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.958714    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28bc3dd5-b6ea-46ef-8142-d202775c9036-tls-certs\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.963496 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.963441    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28bc3dd5-b6ea-46ef-8142-d202775c9036-tls-certs\") pod \"model-serving-api-54d89bd5fc-klgvp\" (UID: \"28bc3dd5-b6ea-46ef-8142-d202775c9036\") " pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:00.988413 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:00.988135    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:01.174855 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:01.174818    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:01.231288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:01.231259    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"]
Jul 27 19:34:01.233884 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:01.233847    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65e628d_169b_4c41_85b3_23b3abb338c0.slice/crio-e10dc1be84959815e11f76fc31333444173b62b50f043ce2ef454316372ef3b4 WatchSource:0}: Error finding container e10dc1be84959815e11f76fc31333444173b62b50f043ce2ef454316372ef3b4: Status 404 returned error can't find the container with id e10dc1be84959815e11f76fc31333444173b62b50f043ce2ef454316372ef3b4
Jul 27 19:34:01.403590 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:01.403551    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28bc3dd5_b6ea_46ef_8142_d202775c9036.slice/crio-c5d1e34cee2fef952bd5daad0261944e1599b48d6f90a2dfa3020b1d2b1715a5 WatchSource:0}: Error finding container c5d1e34cee2fef952bd5daad0261944e1599b48d6f90a2dfa3020b1d2b1715a5: Status 404 returned error can't find the container with id c5d1e34cee2fef952bd5daad0261944e1599b48d6f90a2dfa3020b1d2b1715a5
Jul 27 19:34:01.404409 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:01.404381    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-klgvp"]
Jul 27 19:34:01.545904 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:01.545855    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp" event={"ID":"28bc3dd5-b6ea-46ef-8142-d202775c9036","Type":"ContainerStarted","Data":"c5d1e34cee2fef952bd5daad0261944e1599b48d6f90a2dfa3020b1d2b1715a5"}
Jul 27 19:34:01.548296 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:01.548264    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22" event={"ID":"e65e628d-169b-4c41-85b3-23b3abb338c0","Type":"ContainerStarted","Data":"e10dc1be84959815e11f76fc31333444173b62b50f043ce2ef454316372ef3b4"}
Jul 27 19:34:04.565825 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:04.565774    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv" event={"ID":"f61a9a12-e963-4935-8e03-b67c0b35f921","Type":"ContainerStarted","Data":"d52e2632e190a87f772e3b4e6452f66db3cd8bc4e4d2389bb50e82515abc8687"}
Jul 27 19:34:04.591213 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:04.591160    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b7cdcf968-mgjtv" podStartSLOduration=1.804123803 podStartE2EDuration="5.591139057s" totalImagesPullingTime="3.787015254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:04.587862065 +0000 UTC m=+463.245571593" watchObservedRunningTime="2026-07-27 19:34:04.591139057 +0000 UTC m=+463.248848585"
Jul 27 19:34:05.134606 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.134347    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:34:05.134606 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.134443    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:34:05.574124 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.573852    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp" event={"ID":"28bc3dd5-b6ea-46ef-8142-d202775c9036","Type":"ContainerStarted","Data":"0d74537e8a466e27dd78eb5d2b3cc9c015e5092f6267d9ef5aae448c2fcab429"}
Jul 27 19:34:05.574124 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.573970    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:05.575917 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.575881    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s" event={"ID":"693be74d-32c6-4fa1-ac4b-ce702bfc3910","Type":"ContainerStarted","Data":"dddb2c454a5dfaf27f8735cab290284961dba2393034f3c7e318aa8c5d794c18"}
Jul 27 19:34:05.576094 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.576067    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:34:05.578484 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.578432    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22" event={"ID":"e65e628d-169b-4c41-85b3-23b3abb338c0","Type":"ContainerStarted","Data":"1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4"}
Jul 27 19:34:05.578645 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.578628    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:05.592473 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.592414    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp" podStartSLOduration=1.898610622 podStartE2EDuration="5.592397672s" totalImagesPullingTime="3.69378705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:05.590637589 +0000 UTC m=+464.248347132" watchObservedRunningTime="2026-07-27 19:34:05.592397672 +0000 UTC m=+464.250107201"
Jul 27 19:34:05.613520 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.613457    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22" podStartSLOduration=1.715959217 podStartE2EDuration="5.613443909s" totalImagesPullingTime="3.897484692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:05.612364229 +0000 UTC m=+464.270073758" watchObservedRunningTime="2026-07-27 19:34:05.613443909 +0000 UTC m=+464.271153437"
Jul 27 19:34:05.632247 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:05.632185    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s" podStartSLOduration=3.060822526 podStartE2EDuration="6.632164796s" totalImagesPullingTime="3.57134227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:05.629348842 +0000 UTC m=+464.287058373" watchObservedRunningTime="2026-07-27 19:34:05.632164796 +0000 UTC m=+464.289874328"
Jul 27 19:34:06.583783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:06.583753    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:10.863764 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.863717    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"]
Jul 27 19:34:10.866623 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.866607    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:10.869288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.869260    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 27 19:34:10.869374 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.869332    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rr2f5\""
Jul 27 19:34:10.875193 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.875173    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"]
Jul 27 19:34:10.877767 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.877745    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:10.879343 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.879280    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"]
Jul 27 19:34:10.880378 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.880354    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 27 19:34:10.880494 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.880410    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tshmb\""
Jul 27 19:34:10.890707 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.890682    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"]
Jul 27 19:34:10.953326 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.953295    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59vsm\" (UniqueName: \"kubernetes.io/projected/8cbec7e3-7c5d-4624-a075-07fdf2561442-kube-api-access-59vsm\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:10.953466 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.953330    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cbec7e3-7c5d-4624-a075-07fdf2561442-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:10.953466 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.953367    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89pjg\" (UniqueName: \"kubernetes.io/projected/440c080f-3966-451b-912b-9d8881cbddb6-kube-api-access-89pjg\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:10.953466 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:10.953385    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/440c080f-3966-451b-912b-9d8881cbddb6-cert\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.054115 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.054082    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59vsm\" (UniqueName: \"kubernetes.io/projected/8cbec7e3-7c5d-4624-a075-07fdf2561442-kube-api-access-59vsm\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:11.054296 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.054122    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cbec7e3-7c5d-4624-a075-07fdf2561442-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:11.054296 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:11.054229    2544 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 27 19:34:11.054296 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.054257    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89pjg\" (UniqueName: \"kubernetes.io/projected/440c080f-3966-451b-912b-9d8881cbddb6-kube-api-access-89pjg\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.054296 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:11.054286    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbec7e3-7c5d-4624-a075-07fdf2561442-cert podName:8cbec7e3-7c5d-4624-a075-07fdf2561442 nodeName:}" failed. No retries permitted until 2026-07-27 19:34:11.554268382 +0000 UTC m=+470.211977890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8cbec7e3-7c5d-4624-a075-07fdf2561442-cert") pod "llmisvc-controller-manager-777c6bd7d5-gwwfj" (UID: "8cbec7e3-7c5d-4624-a075-07fdf2561442") : secret "llmisvc-webhook-server-cert" not found
Jul 27 19:34:11.054517 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.054321    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/440c080f-3966-451b-912b-9d8881cbddb6-cert\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.054517 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:11.054449    2544 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 27 19:34:11.054517 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:11.054489    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/440c080f-3966-451b-912b-9d8881cbddb6-cert podName:440c080f-3966-451b-912b-9d8881cbddb6 nodeName:}" failed. No retries permitted until 2026-07-27 19:34:11.554477818 +0000 UTC m=+470.212187326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/440c080f-3966-451b-912b-9d8881cbddb6-cert") pod "kserve-controller-manager-d6f4b65b8-g82mq" (UID: "440c080f-3966-451b-912b-9d8881cbddb6") : secret "kserve-webhook-server-cert" not found
Jul 27 19:34:11.062939 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.062915    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59vsm\" (UniqueName: \"kubernetes.io/projected/8cbec7e3-7c5d-4624-a075-07fdf2561442-kube-api-access-59vsm\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:11.063330 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.063308    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89pjg\" (UniqueName: \"kubernetes.io/projected/440c080f-3966-451b-912b-9d8881cbddb6-kube-api-access-89pjg\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.559357 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.559323    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cbec7e3-7c5d-4624-a075-07fdf2561442-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:11.559528 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.559380    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/440c080f-3966-451b-912b-9d8881cbddb6-cert\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.561705 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.561674    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8cbec7e3-7c5d-4624-a075-07fdf2561442-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-gwwfj\" (UID: \"8cbec7e3-7c5d-4624-a075-07fdf2561442\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:11.561843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.561781    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/440c080f-3966-451b-912b-9d8881cbddb6-cert\") pod \"kserve-controller-manager-d6f4b65b8-g82mq\" (UID: \"440c080f-3966-451b-912b-9d8881cbddb6\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.776744 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.776693    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:11.787976 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.787948    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:11.933894 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.933841    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"]
Jul 27 19:34:11.936224 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:11.936193    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440c080f_3966_451b_912b_9d8881cbddb6.slice/crio-35ba35bdafe058f07e39de00f73148bd18e365a272ba18a9fa7d95b09fcc59f0 WatchSource:0}: Error finding container 35ba35bdafe058f07e39de00f73148bd18e365a272ba18a9fa7d95b09fcc59f0: Status 404 returned error can't find the container with id 35ba35bdafe058f07e39de00f73148bd18e365a272ba18a9fa7d95b09fcc59f0
Jul 27 19:34:11.954010 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:11.953991    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"]
Jul 27 19:34:11.956098 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:11.956076    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8cbec7e3_7c5d_4624_a075_07fdf2561442.slice/crio-5c71c9f35ad7982239097589c05476ef35e65e12f6254fa1ec21067d7a73a277 WatchSource:0}: Error finding container 5c71c9f35ad7982239097589c05476ef35e65e12f6254fa1ec21067d7a73a277: Status 404 returned error can't find the container with id 5c71c9f35ad7982239097589c05476ef35e65e12f6254fa1ec21067d7a73a277
Jul 27 19:34:12.609602 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:12.609562    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq" event={"ID":"440c080f-3966-451b-912b-9d8881cbddb6","Type":"ContainerStarted","Data":"35ba35bdafe058f07e39de00f73148bd18e365a272ba18a9fa7d95b09fcc59f0"}
Jul 27 19:34:12.611217 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:12.611181    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj" event={"ID":"8cbec7e3-7c5d-4624-a075-07fdf2561442","Type":"ContainerStarted","Data":"5c71c9f35ad7982239097589c05476ef35e65e12f6254fa1ec21067d7a73a277"}
Jul 27 19:34:13.109794 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.109759    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"]
Jul 27 19:34:13.110192 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.109961    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22" podUID="e65e628d-169b-4c41-85b3-23b3abb338c0" containerName="discovery" containerID="cri-o://1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4" gracePeriod=30
Jul 27 19:34:13.409718 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.409688    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:13.475290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475216    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-token\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.475290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475282    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjfwc\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-kube-api-access-rjfwc\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.475527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475350    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-kubeconfig\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.475527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475401    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65e628d-169b-4c41-85b3-23b3abb338c0-local-certs\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.475527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475436    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-dns-cert\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.475527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475464    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-cacerts\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.475527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.475495    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-ca-configmap\") pod \"e65e628d-169b-4c41-85b3-23b3abb338c0\" (UID: \"e65e628d-169b-4c41-85b3-23b3abb338c0\") "
Jul 27 19:34:13.476494 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.476438    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:34:13.478812 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.478713    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:34:13.478950 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.478889    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:34:13.479053 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.478986    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-cacerts" (OuterVolumeSpecName: "cacerts") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:34:13.479321 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.479290    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:34:13.479461 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.479385    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65e628d-169b-4c41-85b3-23b3abb338c0-local-certs" (OuterVolumeSpecName: "local-certs") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:34:13.479461 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.479420    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-kube-api-access-rjfwc" (OuterVolumeSpecName: "kube-api-access-rjfwc") pod "e65e628d-169b-4c41-85b3-23b3abb338c0" (UID: "e65e628d-169b-4c41-85b3-23b3abb338c0"). InnerVolumeSpecName "kube-api-access-rjfwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:34:13.576572 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576529    2544 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-kubeconfig\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.576572 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576571    2544 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65e628d-169b-4c41-85b3-23b3abb338c0-local-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.576802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576586    2544 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-dns-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.576802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576599    2544 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e65e628d-169b-4c41-85b3-23b3abb338c0-cacerts\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.576802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576612    2544 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-csr-ca-configmap\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.576802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576624    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.576802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.576632    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjfwc\" (UniqueName: \"kubernetes.io/projected/e65e628d-169b-4c41-85b3-23b3abb338c0-kube-api-access-rjfwc\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:34:13.619506 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.619467    2544 generic.go:356] "Generic (PLEG): container finished" podID="e65e628d-169b-4c41-85b3-23b3abb338c0" containerID="1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4" exitCode=0
Jul 27 19:34:13.619656 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.619530    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22" event={"ID":"e65e628d-169b-4c41-85b3-23b3abb338c0","Type":"ContainerDied","Data":"1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4"}
Jul 27 19:34:13.619656 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.619537    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"
Jul 27 19:34:13.619656 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.619610    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22" event={"ID":"e65e628d-169b-4c41-85b3-23b3abb338c0","Type":"ContainerDied","Data":"e10dc1be84959815e11f76fc31333444173b62b50f043ce2ef454316372ef3b4"}
Jul 27 19:34:13.619656 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.619638    2544 scope.go:117] "RemoveContainer" containerID="1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4"
Jul 27 19:34:13.647052 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.646972    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"]
Jul 27 19:34:13.651397 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.651374    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-clh22"]
Jul 27 19:34:13.842416 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:13.842078    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e65e628d-169b-4c41-85b3-23b3abb338c0" path="/var/lib/kubelet/pods/e65e628d-169b-4c41-85b3-23b3abb338c0/volumes"
Jul 27 19:34:14.565494 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:14.565257    2544 scope.go:117] "RemoveContainer" containerID="1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4"
Jul 27 19:34:14.565846 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:34:14.565601    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4\": container with ID starting with 1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4 not found: ID does not exist" containerID="1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4"
Jul 27 19:34:14.565846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:14.565630    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4"} err="failed to get container status \"1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4\": rpc error: code = NotFound desc = could not find container \"1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4\": container with ID starting with 1c9a3451e38c8631095727f9878829ed5164df817ab1907232d07507cfe412a4 not found: ID does not exist"
Jul 27 19:34:15.630940 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:15.630907    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj" event={"ID":"8cbec7e3-7c5d-4624-a075-07fdf2561442","Type":"ContainerStarted","Data":"d8e08f1aef72a948d4913909877173140f5a09f315ab6f4056c934dc0bb2a00f"}
Jul 27 19:34:15.631381 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:15.631076    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:15.632395 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:15.632371    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq" event={"ID":"440c080f-3966-451b-912b-9d8881cbddb6","Type":"ContainerStarted","Data":"32337f52add0f45f961888ba98a94f715c788b5bb69c9d4d0419c4c2624d5d7f"}
Jul 27 19:34:15.632486 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:15.632469    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:15.649848 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:15.649796    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj" podStartSLOduration=3.028936037 podStartE2EDuration="5.649782192s" totalImagesPullingTime="2.620846155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:15.648562602 +0000 UTC m=+474.306272124" watchObservedRunningTime="2026-07-27 19:34:15.649782192 +0000 UTC m=+474.307491720"
Jul 27 19:34:15.664023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:15.663984    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq" podStartSLOduration=3.027641533 podStartE2EDuration="5.663970353s" totalImagesPullingTime="2.63632882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:34:15.663283528 +0000 UTC m=+474.320993056" watchObservedRunningTime="2026-07-27 19:34:15.663970353 +0000 UTC m=+474.321679882"
Jul 27 19:34:16.584179 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:16.584152    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ntf9s"
Jul 27 19:34:16.585957 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:16.585936    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-klgvp"
Jul 27 19:34:46.639858 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:46.639824    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-gwwfj"
Jul 27 19:34:46.643103 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:46.643083    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-g82mq"
Jul 27 19:34:57.852527 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.852495    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lw87x"]
Jul 27 19:34:57.852902 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.852867    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e65e628d-169b-4c41-85b3-23b3abb338c0" containerName="discovery"
Jul 27 19:34:57.852902 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.852881    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="e65e628d-169b-4c41-85b3-23b3abb338c0" containerName="discovery"
Jul 27 19:34:57.852973 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.852934    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="e65e628d-169b-4c41-85b3-23b3abb338c0" containerName="discovery"
Jul 27 19:34:57.857854 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.857832    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:34:57.860574 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.860477    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 19:34:57.860574 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.860540    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 19:34:57.860781 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.860540    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pjpkv\""
Jul 27 19:34:57.866664 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.866639    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lw87x"]
Jul 27 19:34:57.950798 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:57.950767    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6-kube-api-access-7zlnw\") pod \"kuadrant-operator-catalog-lw87x\" (UID: \"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6\") " pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:34:58.051350 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.051320    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6-kube-api-access-7zlnw\") pod \"kuadrant-operator-catalog-lw87x\" (UID: \"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6\") " pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:34:58.060379 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.060355    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6-kube-api-access-7zlnw\") pod \"kuadrant-operator-catalog-lw87x\" (UID: \"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6\") " pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:34:58.168660 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.168631    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:34:58.220159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.220117    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lw87x"]
Jul 27 19:34:58.301606 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.301576    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lw87x"]
Jul 27 19:34:58.306300 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:58.306277    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e40e9b_ba72_4749_90dd_c9a8a3f78ee6.slice/crio-d6b49d4bfad11e68887fb4ef363a326af1005757b20094fb134cdfd6b607107b WatchSource:0}: Error finding container d6b49d4bfad11e68887fb4ef363a326af1005757b20094fb134cdfd6b607107b: Status 404 returned error can't find the container with id d6b49d4bfad11e68887fb4ef363a326af1005757b20094fb134cdfd6b607107b
Jul 27 19:34:58.431497 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.431427    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5k95s"]
Jul 27 19:34:58.434428 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.434402    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:34:58.441140 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.441116    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5k95s"]
Jul 27 19:34:58.455521 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.455497    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5h2x\" (UniqueName: \"kubernetes.io/projected/49a9e7ab-4829-44b4-a48e-2d478238c739-kube-api-access-h5h2x\") pod \"kuadrant-operator-catalog-5k95s\" (UID: \"49a9e7ab-4829-44b4-a48e-2d478238c739\") " pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:34:58.556826 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.556794    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5h2x\" (UniqueName: \"kubernetes.io/projected/49a9e7ab-4829-44b4-a48e-2d478238c739-kube-api-access-h5h2x\") pod \"kuadrant-operator-catalog-5k95s\" (UID: \"49a9e7ab-4829-44b4-a48e-2d478238c739\") " pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:34:58.564923 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.564893    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5h2x\" (UniqueName: \"kubernetes.io/projected/49a9e7ab-4829-44b4-a48e-2d478238c739-kube-api-access-h5h2x\") pod \"kuadrant-operator-catalog-5k95s\" (UID: \"49a9e7ab-4829-44b4-a48e-2d478238c739\") " pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:34:58.743888 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.743802    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:34:58.812504 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.812475    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lw87x" event={"ID":"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6","Type":"ContainerStarted","Data":"d6b49d4bfad11e68887fb4ef363a326af1005757b20094fb134cdfd6b607107b"}
Jul 27 19:34:58.871785 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:58.871635    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5k95s"]
Jul 27 19:34:58.874205 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:34:58.874179    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49a9e7ab_4829_44b4_a48e_2d478238c739.slice/crio-e542d1f5b4a03a9539183f71de4370f5a9155a2c398df4a2f79ecfdf869acfdf WatchSource:0}: Error finding container e542d1f5b4a03a9539183f71de4370f5a9155a2c398df4a2f79ecfdf869acfdf: Status 404 returned error can't find the container with id e542d1f5b4a03a9539183f71de4370f5a9155a2c398df4a2f79ecfdf869acfdf
Jul 27 19:34:59.818141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:34:59.818104    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5k95s" event={"ID":"49a9e7ab-4829-44b4-a48e-2d478238c739","Type":"ContainerStarted","Data":"e542d1f5b4a03a9539183f71de4370f5a9155a2c398df4a2f79ecfdf869acfdf"}
Jul 27 19:35:00.823742 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:00.823705    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lw87x" event={"ID":"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6","Type":"ContainerStarted","Data":"a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28"}
Jul 27 19:35:00.824153 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:00.823800    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-lw87x" podUID="04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" containerName="registry-server" containerID="cri-o://a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28" gracePeriod=2
Jul 27 19:35:00.825290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:00.825264    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5k95s" event={"ID":"49a9e7ab-4829-44b4-a48e-2d478238c739","Type":"ContainerStarted","Data":"8cf8de00f83ec08ca6d23088b4b1f3f7ad3b1472bfc2985fc8000078e8afa8df"}
Jul 27 19:35:00.839713 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:00.839668    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lw87x" podStartSLOduration=1.8094843900000002 podStartE2EDuration="3.839650456s" totalImagesPullingTime="2.030166066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:00.837726016 +0000 UTC m=+519.495435544" watchObservedRunningTime="2026-07-27 19:35:00.839650456 +0000 UTC m=+519.497359985"
Jul 27 19:35:00.852357 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:00.852316    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5k95s" podStartSLOduration=1.389192759 podStartE2EDuration="2.852302569s" totalImagesPullingTime="1.46310981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:00.851037871 +0000 UTC m=+519.508747404" watchObservedRunningTime="2026-07-27 19:35:00.852302569 +0000 UTC m=+519.510012097"
Jul 27 19:35:01.070913 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.070887    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:35:01.181825 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.181795    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6-kube-api-access-7zlnw\") pod \"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6\" (UID: \"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6\") "
Jul 27 19:35:01.183865 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.183845    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6-kube-api-access-7zlnw" (OuterVolumeSpecName: "kube-api-access-7zlnw") pod "04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" (UID: "04e40e9b-ba72-4749-90dd-c9a8a3f78ee6"). InnerVolumeSpecName "kube-api-access-7zlnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:01.282543 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.282508    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6-kube-api-access-7zlnw\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:35:01.830634 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.830595    2544 generic.go:356] "Generic (PLEG): container finished" podID="04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" containerID="a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28" exitCode=0
Jul 27 19:35:01.831104 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.830642    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lw87x"
Jul 27 19:35:01.831104 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.830673    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lw87x" event={"ID":"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6","Type":"ContainerDied","Data":"a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28"}
Jul 27 19:35:01.831104 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.830720    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lw87x" event={"ID":"04e40e9b-ba72-4749-90dd-c9a8a3f78ee6","Type":"ContainerDied","Data":"d6b49d4bfad11e68887fb4ef363a326af1005757b20094fb134cdfd6b607107b"}
Jul 27 19:35:01.831104 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.830758    2544 scope.go:117] "RemoveContainer" containerID="a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28"
Jul 27 19:35:01.840082 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.840067    2544 scope.go:117] "RemoveContainer" containerID="a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28"
Jul 27 19:35:01.840329 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:01.840312    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28\": container with ID starting with a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28 not found: ID does not exist" containerID="a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28"
Jul 27 19:35:01.840384 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.840338    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28"} err="failed to get container status \"a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28\": rpc error: code = NotFound desc = could not find container \"a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28\": container with ID starting with a5746e8aa506b0a3e5ae738a5e15f6cb2d6dace68a3a18da004217ad0aa21a28 not found: ID does not exist"
Jul 27 19:35:01.850952 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.850931    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lw87x"]
Jul 27 19:35:01.854407 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:01.854388    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lw87x"]
Jul 27 19:35:03.839887 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:03.839854    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" path="/var/lib/kubelet/pods/04e40e9b-ba72-4749-90dd-c9a8a3f78ee6/volumes"
Jul 27 19:35:08.744325 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:08.744293    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:35:08.744724 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:08.744377    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:35:08.765519 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:08.765492    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:35:08.885297 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:08.885269    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5k95s"
Jul 27 19:35:22.525555 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.525519    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"]
Jul 27 19:35:22.525982 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.525871    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" containerName="registry-server"
Jul 27 19:35:22.525982 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.525883    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" containerName="registry-server"
Jul 27 19:35:22.525982 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.525938    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="04e40e9b-ba72-4749-90dd-c9a8a3f78ee6" containerName="registry-server"
Jul 27 19:35:22.530290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.530273    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:22.532774 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.532747    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-svq7w\""
Jul 27 19:35:22.537620 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.537597    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"]
Jul 27 19:35:22.666771 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.666715    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpjxd\" (UniqueName: \"kubernetes.io/projected/5f8b8668-ba34-4fb1-8b04-297fd5e68967-kube-api-access-dpjxd\") pod \"limitador-operator-controller-manager-859d7b78bd-hsrn7\" (UID: \"5f8b8668-ba34-4fb1-8b04-297fd5e68967\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:22.767659 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.767622    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpjxd\" (UniqueName: \"kubernetes.io/projected/5f8b8668-ba34-4fb1-8b04-297fd5e68967-kube-api-access-dpjxd\") pod \"limitador-operator-controller-manager-859d7b78bd-hsrn7\" (UID: \"5f8b8668-ba34-4fb1-8b04-297fd5e68967\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:22.779122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.779054    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpjxd\" (UniqueName: \"kubernetes.io/projected/5f8b8668-ba34-4fb1-8b04-297fd5e68967-kube-api-access-dpjxd\") pod \"limitador-operator-controller-manager-859d7b78bd-hsrn7\" (UID: \"5f8b8668-ba34-4fb1-8b04-297fd5e68967\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:22.841016 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:22.840984    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:23.010030 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:35:23.010003    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f8b8668_ba34_4fb1_8b04_297fd5e68967.slice/crio-02b5027da703de1bdf1bf93b8b4d230970b14665e5dde30dcb913bd459498625 WatchSource:0}: Error finding container 02b5027da703de1bdf1bf93b8b4d230970b14665e5dde30dcb913bd459498625: Status 404 returned error can't find the container with id 02b5027da703de1bdf1bf93b8b4d230970b14665e5dde30dcb913bd459498625
Jul 27 19:35:23.012276 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:23.012256    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"]
Jul 27 19:35:23.928721 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:23.928675    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7" event={"ID":"5f8b8668-ba34-4fb1-8b04-297fd5e68967","Type":"ContainerStarted","Data":"02b5027da703de1bdf1bf93b8b4d230970b14665e5dde30dcb913bd459498625"}
Jul 27 19:35:24.935361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:24.935319    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7" event={"ID":"5f8b8668-ba34-4fb1-8b04-297fd5e68967","Type":"ContainerStarted","Data":"49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec"}
Jul 27 19:35:24.935754 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:24.935398    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:24.953724 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:24.953677    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7" podStartSLOduration=1.876975367 podStartE2EDuration="2.953663696s" totalImagesPullingTime="1.076688329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:24.95261276 +0000 UTC m=+543.610322289" watchObservedRunningTime="2026-07-27 19:35:24.953663696 +0000 UTC m=+543.611373227"
Jul 27 19:35:28.097002 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.096968    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"]
Jul 27 19:35:28.100209 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.100192    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.102980 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.102962    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-htjbz\""
Jul 27 19:35:28.111752 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.111709    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"]
Jul 27 19:35:28.218718 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.218684    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.218895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.218795    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzn99\" (UniqueName: \"kubernetes.io/projected/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-kube-api-access-wzn99\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.319868 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.319838    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzn99\" (UniqueName: \"kubernetes.io/projected/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-kube-api-access-wzn99\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.319995 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.319887    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.320221 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.320206    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.329047 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.329021    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzn99\" (UniqueName: \"kubernetes.io/projected/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-kube-api-access-wzn99\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.409899 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.409867    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:28.544012 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.543981    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"]
Jul 27 19:35:28.547499 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:35:28.547472    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03654cee_7531_4ee3_9fc0_c9eed9b7a14a.slice/crio-c5a0cad16a365f5958d707e3778b67ffe09defe6864c54303c7ee18b3c2463c1 WatchSource:0}: Error finding container c5a0cad16a365f5958d707e3778b67ffe09defe6864c54303c7ee18b3c2463c1: Status 404 returned error can't find the container with id c5a0cad16a365f5958d707e3778b67ffe09defe6864c54303c7ee18b3c2463c1
Jul 27 19:35:28.957165 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:28.957100    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj" event={"ID":"03654cee-7531-4ee3-9fc0-c9eed9b7a14a","Type":"ContainerStarted","Data":"c5a0cad16a365f5958d707e3778b67ffe09defe6864c54303c7ee18b3c2463c1"}
Jul 27 19:35:31.887478 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:31.887447    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"]
Jul 27 19:35:31.891185 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:31.891163    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:31.893789 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:31.893763    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-krkjl\""
Jul 27 19:35:31.903236 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:31.903203    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"]
Jul 27 19:35:31.950419 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:31.950384    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ws9c\" (UniqueName: \"kubernetes.io/projected/ae5f9549-f879-4175-91e2-8d948532304d-kube-api-access-7ws9c\") pod \"authorino-operator-75cc8d5cb7-ldrk7\" (UID: \"ae5f9549-f879-4175-91e2-8d948532304d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:32.051334 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.051286    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ws9c\" (UniqueName: \"kubernetes.io/projected/ae5f9549-f879-4175-91e2-8d948532304d-kube-api-access-7ws9c\") pod \"authorino-operator-75cc8d5cb7-ldrk7\" (UID: \"ae5f9549-f879-4175-91e2-8d948532304d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:32.061130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.061099    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ws9c\" (UniqueName: \"kubernetes.io/projected/ae5f9549-f879-4175-91e2-8d948532304d-kube-api-access-7ws9c\") pod \"authorino-operator-75cc8d5cb7-ldrk7\" (UID: \"ae5f9549-f879-4175-91e2-8d948532304d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:32.205688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.205668    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:32.343246 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.343132    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"]
Jul 27 19:35:32.346105 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:35:32.346077    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5f9549_f879_4175_91e2_8d948532304d.slice/crio-d52867aaac82f5e4a341dffa680651be722030dc95560ca2eb97455af52fa135 WatchSource:0}: Error finding container d52867aaac82f5e4a341dffa680651be722030dc95560ca2eb97455af52fa135: Status 404 returned error can't find the container with id d52867aaac82f5e4a341dffa680651be722030dc95560ca2eb97455af52fa135
Jul 27 19:35:32.980242 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.980201    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7" event={"ID":"ae5f9549-f879-4175-91e2-8d948532304d","Type":"ContainerStarted","Data":"d52867aaac82f5e4a341dffa680651be722030dc95560ca2eb97455af52fa135"}
Jul 27 19:35:32.982645 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.982460    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj" event={"ID":"03654cee-7531-4ee3-9fc0-c9eed9b7a14a","Type":"ContainerStarted","Data":"4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4"}
Jul 27 19:35:32.982645 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:32.982550    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:33.005012 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:33.004956    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj" podStartSLOduration=1.407222338 podStartE2EDuration="5.00493539s" totalImagesPullingTime="3.597713052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:33.003945484 +0000 UTC m=+551.661655009" watchObservedRunningTime="2026-07-27 19:35:33.00493539 +0000 UTC m=+551.662644921"
Jul 27 19:35:34.995837 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:34.995797    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7" event={"ID":"ae5f9549-f879-4175-91e2-8d948532304d","Type":"ContainerStarted","Data":"117d03814a974921984ea51a225e697f52a4afc1327db0c0e534fe3bdc99fa78"}
Jul 27 19:35:34.996218 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:34.995963    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:35.016411 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:35.016364    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7" podStartSLOduration=1.7717284709999999 podStartE2EDuration="4.01634881s" totalImagesPullingTime="2.244620339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:35.014670723 +0000 UTC m=+553.672380249" watchObservedRunningTime="2026-07-27 19:35:35.01634881 +0000 UTC m=+553.674058338"
Jul 27 19:35:35.942415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:35.942387    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:43.991604 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:43.991568    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:45.674101 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.674071    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"]
Jul 27 19:35:45.674492 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.674254    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" containerName="manager" containerID="cri-o://4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4" gracePeriod=2
Jul 27 19:35:45.678865 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.678837    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"]
Jul 27 19:35:45.696566 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.696537    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"]
Jul 27 19:35:45.696850 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.696791    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" containerName="manager" containerID="cri-o://49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec" gracePeriod=2
Jul 27 19:35:45.701798 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.701773    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"]
Jul 27 19:35:45.702654 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.702235    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" containerName="manager"
Jul 27 19:35:45.702654 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.702253    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" containerName="manager"
Jul 27 19:35:45.702654 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.702372    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" containerName="manager"
Jul 27 19:35:45.704304 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.704286    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.710315 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.710295    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"]
Jul 27 19:35:45.717073 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:45.717043    2544 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5f8b8668-ba34-4fb1-8b04-297fd5e68967\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-hsrn7\": pods \"limitador-operator-controller-manager-859d7b78bd-hsrn7\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:45.719707 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:45.719678    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:45.720753 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.720708    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"]
Jul 27 19:35:45.723000 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.722979    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"]
Jul 27 19:35:45.723366 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.723348    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" containerName="manager"
Jul 27 19:35:45.723479 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.723461    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" containerName="manager"
Jul 27 19:35:45.723591 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.723583    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" containerName="manager"
Jul 27 19:35:45.725488 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.725473    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:35:45.735713 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.735693    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"]
Jul 27 19:35:45.745665 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:45.745637    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hsrn7\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:45.747088 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.747064    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72nt7\" (UniqueName: \"kubernetes.io/projected/31bba75d-070f-43b6-985e-fdbe76fff831-kube-api-access-72nt7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-48wlr\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.747161 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.747126    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31bba75d-070f-43b6-985e-fdbe76fff831-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-48wlr\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.747619 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:45.747590    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:45.765086 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:45.765061    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hsrn7\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:45.767087 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:45.767065    2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jhctj\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:45.847705 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.847672    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72nt7\" (UniqueName: \"kubernetes.io/projected/31bba75d-070f-43b6-985e-fdbe76fff831-kube-api-access-72nt7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-48wlr\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.847705 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.847710    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn7cw\" (UniqueName: \"kubernetes.io/projected/436f90f9-8233-4074-b687-2e6bf5f3861a-kube-api-access-qn7cw\") pod \"limitador-operator-controller-manager-859d7b78bd-th5td\" (UID: \"436f90f9-8233-4074-b687-2e6bf5f3861a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:35:45.847904 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.847761    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31bba75d-070f-43b6-985e-fdbe76fff831-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-48wlr\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.848102 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.848085    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31bba75d-070f-43b6-985e-fdbe76fff831-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-48wlr\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.856574 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.856538    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72nt7\" (UniqueName: \"kubernetes.io/projected/31bba75d-070f-43b6-985e-fdbe76fff831-kube-api-access-72nt7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-48wlr\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:45.940886 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.940864    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:45.943465 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.943445    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:45.948147 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.948128    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpjxd\" (UniqueName: \"kubernetes.io/projected/5f8b8668-ba34-4fb1-8b04-297fd5e68967-kube-api-access-dpjxd\") pod \"5f8b8668-ba34-4fb1-8b04-297fd5e68967\" (UID: \"5f8b8668-ba34-4fb1-8b04-297fd5e68967\") "
Jul 27 19:35:45.948240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.948164    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-extensions-socket-volume\") pod \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") "
Jul 27 19:35:45.948240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.948182    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzn99\" (UniqueName: \"kubernetes.io/projected/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-kube-api-access-wzn99\") pod \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\" (UID: \"03654cee-7531-4ee3-9fc0-c9eed9b7a14a\") "
Jul 27 19:35:45.948364 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.948349    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn7cw\" (UniqueName: \"kubernetes.io/projected/436f90f9-8233-4074-b687-2e6bf5f3861a-kube-api-access-qn7cw\") pod \"limitador-operator-controller-manager-859d7b78bd-th5td\" (UID: \"436f90f9-8233-4074-b687-2e6bf5f3861a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:35:45.948676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.948641    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03654cee-7531-4ee3-9fc0-c9eed9b7a14a" (UID: "03654cee-7531-4ee3-9fc0-c9eed9b7a14a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:35:45.950075 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.950058    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8b8668-ba34-4fb1-8b04-297fd5e68967-kube-api-access-dpjxd" (OuterVolumeSpecName: "kube-api-access-dpjxd") pod "5f8b8668-ba34-4fb1-8b04-297fd5e68967" (UID: "5f8b8668-ba34-4fb1-8b04-297fd5e68967"). InnerVolumeSpecName "kube-api-access-dpjxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:45.950163 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.950145    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-kube-api-access-wzn99" (OuterVolumeSpecName: "kube-api-access-wzn99") pod "03654cee-7531-4ee3-9fc0-c9eed9b7a14a" (UID: "03654cee-7531-4ee3-9fc0-c9eed9b7a14a"). InnerVolumeSpecName "kube-api-access-wzn99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:35:45.957022 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:45.957001    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn7cw\" (UniqueName: \"kubernetes.io/projected/436f90f9-8233-4074-b687-2e6bf5f3861a-kube-api-access-qn7cw\") pod \"limitador-operator-controller-manager-859d7b78bd-th5td\" (UID: \"436f90f9-8233-4074-b687-2e6bf5f3861a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:35:46.002815 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.002792    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ldrk7"
Jul 27 19:35:46.043720 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.043694    2544 generic.go:356] "Generic (PLEG): container finished" podID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" containerID="4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4" exitCode=0
Jul 27 19:35:46.043871 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.043763    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jhctj"
Jul 27 19:35:46.043871 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.043793    2544 scope.go:117] "RemoveContainer" containerID="4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4"
Jul 27 19:35:46.045779 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.045526    2544 generic.go:356] "Generic (PLEG): container finished" podID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" containerID="49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec" exitCode=0
Jul 27 19:35:46.045779 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.045591    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hsrn7"
Jul 27 19:35:46.048915 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.048882    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpjxd\" (UniqueName: \"kubernetes.io/projected/5f8b8668-ba34-4fb1-8b04-297fd5e68967-kube-api-access-dpjxd\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:35:46.048915 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.048908    2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:35:46.049093 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.048922    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzn99\" (UniqueName: \"kubernetes.io/projected/03654cee-7531-4ee3-9fc0-c9eed9b7a14a-kube-api-access-wzn99\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:35:46.054616 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.054508    2544 scope.go:117] "RemoveContainer" containerID="4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4"
Jul 27 19:35:46.054928 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:46.054846    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4\": container with ID starting with 4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4 not found: ID does not exist" containerID="4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4"
Jul 27 19:35:46.055028 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.054941    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4"} err="failed to get container status \"4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4\": rpc error: code = NotFound desc = could not find container \"4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4\": container with ID starting with 4f2db7b78cd8206aaa42824d7cc4084be8ee16cd6b157fa47e2f2296bba802a4 not found: ID does not exist"
Jul 27 19:35:46.055028 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.054965    2544 scope.go:117] "RemoveContainer" containerID="49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec"
Jul 27 19:35:46.068755 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.068714    2544 scope.go:117] "RemoveContainer" containerID="49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec"
Jul 27 19:35:46.069424 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:35:46.069401    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec\": container with ID starting with 49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec not found: ID does not exist" containerID="49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec"
Jul 27 19:35:46.069500 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.069430    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec"} err="failed to get container status \"49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec\": rpc error: code = NotFound desc = could not find container \"49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec\": container with ID starting with 49b52d4f0423fcac5e5cf4fe3445f1781cf6036cb45dd3029cbc12a3fd2791ec not found: ID does not exist"
Jul 27 19:35:46.107771 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.107721    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:46.114220 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.114206    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:35:46.247968 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.247939    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"]
Jul 27 19:35:46.250087 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:35:46.250063    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bba75d_070f_43b6_985e_fdbe76fff831.slice/crio-a96200d4ab114cae94f34ded9b1bd791732add623562db84d81732e7f1984c8e WatchSource:0}: Error finding container a96200d4ab114cae94f34ded9b1bd791732add623562db84d81732e7f1984c8e: Status 404 returned error can't find the container with id a96200d4ab114cae94f34ded9b1bd791732add623562db84d81732e7f1984c8e
Jul 27 19:35:46.266598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:46.266527    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"]
Jul 27 19:35:46.269314 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:35:46.269288    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436f90f9_8233_4074_b687_2e6bf5f3861a.slice/crio-ae4795597e6d5577d05ad4fa39f3cf6f5116806c5716d6464d9ea2e571e74042 WatchSource:0}: Error finding container ae4795597e6d5577d05ad4fa39f3cf6f5116806c5716d6464d9ea2e571e74042: Status 404 returned error can't find the container with id ae4795597e6d5577d05ad4fa39f3cf6f5116806c5716d6464d9ea2e571e74042
Jul 27 19:35:47.051929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.051843    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td" event={"ID":"436f90f9-8233-4074-b687-2e6bf5f3861a","Type":"ContainerStarted","Data":"9a3b2ac8c3c0f17b68a24eff7955ef9632b7a970ee3443b97cff19230f5e528f"}
Jul 27 19:35:47.051929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.051892    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td" event={"ID":"436f90f9-8233-4074-b687-2e6bf5f3861a","Type":"ContainerStarted","Data":"ae4795597e6d5577d05ad4fa39f3cf6f5116806c5716d6464d9ea2e571e74042"}
Jul 27 19:35:47.052334 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.051941    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:35:47.053595 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.053560    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr" event={"ID":"31bba75d-070f-43b6-985e-fdbe76fff831","Type":"ContainerStarted","Data":"16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922"}
Jul 27 19:35:47.053720 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.053608    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr" event={"ID":"31bba75d-070f-43b6-985e-fdbe76fff831","Type":"ContainerStarted","Data":"a96200d4ab114cae94f34ded9b1bd791732add623562db84d81732e7f1984c8e"}
Jul 27 19:35:47.053720 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.053636    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:47.077422 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.077379    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td" podStartSLOduration=2.077365983 podStartE2EDuration="2.077365983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:47.077220285 +0000 UTC m=+565.734929825" watchObservedRunningTime="2026-07-27 19:35:47.077365983 +0000 UTC m=+565.735075511"
Jul 27 19:35:47.112398 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.112339    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr" podStartSLOduration=2.112315309 podStartE2EDuration="2.112315309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:35:47.111513908 +0000 UTC m=+565.769223436" watchObservedRunningTime="2026-07-27 19:35:47.112315309 +0000 UTC m=+565.770024837"
Jul 27 19:35:47.839764 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.839716    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03654cee-7531-4ee3-9fc0-c9eed9b7a14a" path="/var/lib/kubelet/pods/03654cee-7531-4ee3-9fc0-c9eed9b7a14a/volumes"
Jul 27 19:35:47.840085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:47.840072    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8b8668-ba34-4fb1-8b04-297fd5e68967" path="/var/lib/kubelet/pods/5f8b8668-ba34-4fb1-8b04-297fd5e68967/volumes"
Jul 27 19:35:58.061424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:58.061396    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:35:58.061909 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:35:58.061464    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-th5td"
Jul 27 19:36:12.171690 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.171656    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"]
Jul 27 19:36:12.172092 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.171880    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr" podUID="31bba75d-070f-43b6-985e-fdbe76fff831" containerName="manager" containerID="cri-o://16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922" gracePeriod=10
Jul 27 19:36:12.412158 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.412134    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:36:12.464559 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.464494    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72nt7\" (UniqueName: \"kubernetes.io/projected/31bba75d-070f-43b6-985e-fdbe76fff831-kube-api-access-72nt7\") pod \"31bba75d-070f-43b6-985e-fdbe76fff831\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") "
Jul 27 19:36:12.464693 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.464582    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31bba75d-070f-43b6-985e-fdbe76fff831-extensions-socket-volume\") pod \"31bba75d-070f-43b6-985e-fdbe76fff831\" (UID: \"31bba75d-070f-43b6-985e-fdbe76fff831\") "
Jul 27 19:36:12.464973 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.464953    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31bba75d-070f-43b6-985e-fdbe76fff831-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "31bba75d-070f-43b6-985e-fdbe76fff831" (UID: "31bba75d-070f-43b6-985e-fdbe76fff831"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:36:12.466495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.466471    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31bba75d-070f-43b6-985e-fdbe76fff831-kube-api-access-72nt7" (OuterVolumeSpecName: "kube-api-access-72nt7") pod "31bba75d-070f-43b6-985e-fdbe76fff831" (UID: "31bba75d-070f-43b6-985e-fdbe76fff831"). InnerVolumeSpecName "kube-api-access-72nt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:36:12.566080 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.566048    2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31bba75d-070f-43b6-985e-fdbe76fff831-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:36:12.566080 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:12.566077    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72nt7\" (UniqueName: \"kubernetes.io/projected/31bba75d-070f-43b6-985e-fdbe76fff831-kube-api-access-72nt7\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:36:13.177583 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.177552    2544 generic.go:356] "Generic (PLEG): container finished" podID="31bba75d-070f-43b6-985e-fdbe76fff831" containerID="16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922" exitCode=0
Jul 27 19:36:13.178057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.177607    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"
Jul 27 19:36:13.178057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.177633    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr" event={"ID":"31bba75d-070f-43b6-985e-fdbe76fff831","Type":"ContainerDied","Data":"16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922"}
Jul 27 19:36:13.178057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.177742    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr" event={"ID":"31bba75d-070f-43b6-985e-fdbe76fff831","Type":"ContainerDied","Data":"a96200d4ab114cae94f34ded9b1bd791732add623562db84d81732e7f1984c8e"}
Jul 27 19:36:13.178057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.177778    2544 scope.go:117] "RemoveContainer" containerID="16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922"
Jul 27 19:36:13.186711 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.186694    2544 scope.go:117] "RemoveContainer" containerID="16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922"
Jul 27 19:36:13.187010 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:36:13.186991    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922\": container with ID starting with 16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922 not found: ID does not exist" containerID="16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922"
Jul 27 19:36:13.187056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.187018    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922"} err="failed to get container status \"16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922\": rpc error: code = NotFound desc = could not find container \"16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922\": container with ID starting with 16b170ee56a11211ce747277de9d302d3771e0fce67ac541835c4402e239a922 not found: ID does not exist"
Jul 27 19:36:13.200884 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.200858    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"]
Jul 27 19:36:13.202830 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.202808    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-48wlr"]
Jul 27 19:36:13.840676 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:13.840643    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31bba75d-070f-43b6-985e-fdbe76fff831" path="/var/lib/kubelet/pods/31bba75d-070f-43b6-985e-fdbe76fff831/volumes"
Jul 27 19:36:21.724297 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:21.724265    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:36:21.726161 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:21.726139    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:36:28.407462 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.407429    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"]
Jul 27 19:36:28.408885 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.407811    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31bba75d-070f-43b6-985e-fdbe76fff831" containerName="manager"
Jul 27 19:36:28.408885 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.407827    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="31bba75d-070f-43b6-985e-fdbe76fff831" containerName="manager"
Jul 27 19:36:28.408885 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.407907    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="31bba75d-070f-43b6-985e-fdbe76fff831" containerName="manager"
Jul 27 19:36:28.409780 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.409765    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.414169 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.414132    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 27 19:36:28.414404 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.414380    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2srh7\""
Jul 27 19:36:28.423800 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.423775    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"]
Jul 27 19:36:28.506336 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506297    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506336 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506338    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506587 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506367    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506587 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506451    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506587 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506508    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506587 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506530    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqfrr\" (UniqueName: \"kubernetes.io/projected/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-kube-api-access-hqfrr\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506710 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506593    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506710 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506623    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.506710 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.506689    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.607857 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.607821    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608039 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.607867    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608039 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.607911    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608039 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.607933    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqfrr\" (UniqueName: \"kubernetes.io/projected/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-kube-api-access-hqfrr\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608039 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608004    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608039 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608027    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608283 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608056    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608283 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608085    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608283 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608111    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608465 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608423    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608530 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608478    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608580 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608534    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608666 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608642    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.608850 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.608830    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.610319 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.610291    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.610409 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.610392    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.618784 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.618760    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.618935 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.618919    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqfrr\" (UniqueName: \"kubernetes.io/projected/b8a643f3-33b1-4b82-8021-0f632ef6a3a2-kube-api-access-hqfrr\") pod \"maas-default-gateway-openshift-default-5ccb968c66-gxwfr\" (UID: \"b8a643f3-33b1-4b82-8021-0f632ef6a3a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.720805 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.720703    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:28.859155 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:28.859124    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"]
Jul 27 19:36:28.862169 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:36:28.862130    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a643f3_33b1_4b82_8021_0f632ef6a3a2.slice/crio-548fb5c3ba97b037149473d9a717b4e9a0228c0a3a9602b3dd4cead07f716e30 WatchSource:0}: Error finding container 548fb5c3ba97b037149473d9a717b4e9a0228c0a3a9602b3dd4cead07f716e30: Status 404 returned error can't find the container with id 548fb5c3ba97b037149473d9a717b4e9a0228c0a3a9602b3dd4cead07f716e30
Jul 27 19:36:29.249064 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:29.249034    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" event={"ID":"b8a643f3-33b1-4b82-8021-0f632ef6a3a2","Type":"ContainerStarted","Data":"548fb5c3ba97b037149473d9a717b4e9a0228c0a3a9602b3dd4cead07f716e30"}
Jul 27 19:36:31.518495 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:31.518458    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:36:31.518902 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:31.518586    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:36:31.518902 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:31.518632    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:36:32.263849 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:32.263810    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" event={"ID":"b8a643f3-33b1-4b82-8021-0f632ef6a3a2","Type":"ContainerStarted","Data":"227accfc5b54022f6802e3fad3d8c5871c70273150a6e95546528bc33611af15"}
Jul 27 19:36:32.284280 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:32.284233    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" podStartSLOduration=1.630010204 podStartE2EDuration="4.284218724s" totalImagesPullingTime="2.65420852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:32.283472767 +0000 UTC m=+610.941182297" watchObservedRunningTime="2026-07-27 19:36:32.284218724 +0000 UTC m=+610.941928314"
Jul 27 19:36:32.721464 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:32.721429    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:32.726536 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:32.726513    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:33.235752 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.235708    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:33.238361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.238340    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.240793 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.240776    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 27 19:36:33.240858 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.240801    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d5plc\""
Jul 27 19:36:33.248786 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.248762    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:33.268846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.268823    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:33.269849 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.269831    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:36:33.328558 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.328520    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:33.348813 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.348779    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f421af59-3ffd-49ca-96d4-e580236a60b0-config-file\") pod \"limitador-limitador-64c94fd64f-ldwdg\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.348971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.348842    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68gn\" (UniqueName: \"kubernetes.io/projected/f421af59-3ffd-49ca-96d4-e580236a60b0-kube-api-access-f68gn\") pod \"limitador-limitador-64c94fd64f-ldwdg\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.449689 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.449658    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f421af59-3ffd-49ca-96d4-e580236a60b0-config-file\") pod \"limitador-limitador-64c94fd64f-ldwdg\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.449892 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.449698    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68gn\" (UniqueName: \"kubernetes.io/projected/f421af59-3ffd-49ca-96d4-e580236a60b0-kube-api-access-f68gn\") pod \"limitador-limitador-64c94fd64f-ldwdg\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.450397 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.450374    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f421af59-3ffd-49ca-96d4-e580236a60b0-config-file\") pod \"limitador-limitador-64c94fd64f-ldwdg\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.461672 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.461651    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68gn\" (UniqueName: \"kubernetes.io/projected/f421af59-3ffd-49ca-96d4-e580236a60b0-kube-api-access-f68gn\") pod \"limitador-limitador-64c94fd64f-ldwdg\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.548671 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.548589    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:33.676978 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:33.676954    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:33.678660 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:36:33.678628    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf421af59_3ffd_49ca_96d4_e580236a60b0.slice/crio-38ac7aa23db0b118071f907fd0fdd3b069909ad8e3f098982252d3b6458c279b WatchSource:0}: Error finding container 38ac7aa23db0b118071f907fd0fdd3b069909ad8e3f098982252d3b6458c279b: Status 404 returned error can't find the container with id 38ac7aa23db0b118071f907fd0fdd3b069909ad8e3f098982252d3b6458c279b
Jul 27 19:36:34.271824 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.271791    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4w7rw"]
Jul 27 19:36:34.274698 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.274673    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:36:34.276392 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.276368    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg" event={"ID":"f421af59-3ffd-49ca-96d4-e580236a60b0","Type":"ContainerStarted","Data":"38ac7aa23db0b118071f907fd0fdd3b069909ad8e3f098982252d3b6458c279b"}
Jul 27 19:36:34.276969 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.276948    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rjpxs\""
Jul 27 19:36:34.283104 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.283056    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4w7rw"]
Jul 27 19:36:34.459632 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.458958    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v5b8\" (UniqueName: \"kubernetes.io/projected/7808e221-949e-4f47-a12d-b1e71a63857c-kube-api-access-2v5b8\") pod \"authorino-75d556bd77-4w7rw\" (UID: \"7808e221-949e-4f47-a12d-b1e71a63857c\") " pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:36:34.559763 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.559664    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v5b8\" (UniqueName: \"kubernetes.io/projected/7808e221-949e-4f47-a12d-b1e71a63857c-kube-api-access-2v5b8\") pod \"authorino-75d556bd77-4w7rw\" (UID: \"7808e221-949e-4f47-a12d-b1e71a63857c\") " pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:36:34.569026 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.569003    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v5b8\" (UniqueName: \"kubernetes.io/projected/7808e221-949e-4f47-a12d-b1e71a63857c-kube-api-access-2v5b8\") pod \"authorino-75d556bd77-4w7rw\" (UID: \"7808e221-949e-4f47-a12d-b1e71a63857c\") " pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:36:34.587761 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.587712    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:36:34.738641 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.738611    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4w7rw"]
Jul 27 19:36:34.741331 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:36:34.741301    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7808e221_949e_4f47_a12d_b1e71a63857c.slice/crio-09c3a373e6e54c4040a4e40105b6771d241afc9d93100ab31ff3884ce84580ad WatchSource:0}: Error finding container 09c3a373e6e54c4040a4e40105b6771d241afc9d93100ab31ff3884ce84580ad: Status 404 returned error can't find the container with id 09c3a373e6e54c4040a4e40105b6771d241afc9d93100ab31ff3884ce84580ad
Jul 27 19:36:34.743062 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:34.743043    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:36:35.283274 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:35.283233    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4w7rw" event={"ID":"7808e221-949e-4f47-a12d-b1e71a63857c","Type":"ContainerStarted","Data":"09c3a373e6e54c4040a4e40105b6771d241afc9d93100ab31ff3884ce84580ad"}
Jul 27 19:36:37.299240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:37.299204    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg" event={"ID":"f421af59-3ffd-49ca-96d4-e580236a60b0","Type":"ContainerStarted","Data":"670aff8c9c92a4fea992206b93f1f4587f6a9794255a958712ddf66e1905e5e1"}
Jul 27 19:36:37.299710 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:37.299344    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:37.315770 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:37.315697    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg" podStartSLOduration=1.5392733189999999 podStartE2EDuration="4.315680325s" totalImagesPullingTime="2.776407006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:37.314920493 +0000 UTC m=+615.972630022" watchObservedRunningTime="2026-07-27 19:36:37.315680325 +0000 UTC m=+615.973389853"
Jul 27 19:36:39.312439 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:39.312404    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4w7rw" event={"ID":"7808e221-949e-4f47-a12d-b1e71a63857c","Type":"ContainerStarted","Data":"0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600"}
Jul 27 19:36:39.326856 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:39.326800    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4w7rw" podStartSLOduration=1.650677939 podStartE2EDuration="5.326780295s" totalImagesPullingTime="3.676102356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:36:39.326747453 +0000 UTC m=+617.984456977" watchObservedRunningTime="2026-07-27 19:36:39.326780295 +0000 UTC m=+617.984489824"
Jul 27 19:36:48.306393 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:48.306319    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:48.696003 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:48.695969    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:48.696184 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:48.696147    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg" podUID="f421af59-3ffd-49ca-96d4-e580236a60b0" containerName="limitador" containerID="cri-o://670aff8c9c92a4fea992206b93f1f4587f6a9794255a958712ddf66e1905e5e1" gracePeriod=30
Jul 27 19:36:49.358452 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.358425    2544 generic.go:356] "Generic (PLEG): container finished" podID="f421af59-3ffd-49ca-96d4-e580236a60b0" containerID="670aff8c9c92a4fea992206b93f1f4587f6a9794255a958712ddf66e1905e5e1" exitCode=0
Jul 27 19:36:49.358798 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.358503    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg" event={"ID":"f421af59-3ffd-49ca-96d4-e580236a60b0","Type":"ContainerDied","Data":"670aff8c9c92a4fea992206b93f1f4587f6a9794255a958712ddf66e1905e5e1"}
Jul 27 19:36:49.644029 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.643999    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:49.695445 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.695419    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f421af59-3ffd-49ca-96d4-e580236a60b0-config-file\") pod \"f421af59-3ffd-49ca-96d4-e580236a60b0\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") "
Jul 27 19:36:49.695564 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.695459    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f68gn\" (UniqueName: \"kubernetes.io/projected/f421af59-3ffd-49ca-96d4-e580236a60b0-kube-api-access-f68gn\") pod \"f421af59-3ffd-49ca-96d4-e580236a60b0\" (UID: \"f421af59-3ffd-49ca-96d4-e580236a60b0\") "
Jul 27 19:36:49.695830 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.695805    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f421af59-3ffd-49ca-96d4-e580236a60b0-config-file" (OuterVolumeSpecName: "config-file") pod "f421af59-3ffd-49ca-96d4-e580236a60b0" (UID: "f421af59-3ffd-49ca-96d4-e580236a60b0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:36:49.697456 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.697435    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f421af59-3ffd-49ca-96d4-e580236a60b0-kube-api-access-f68gn" (OuterVolumeSpecName: "kube-api-access-f68gn") pod "f421af59-3ffd-49ca-96d4-e580236a60b0" (UID: "f421af59-3ffd-49ca-96d4-e580236a60b0"). InnerVolumeSpecName "kube-api-access-f68gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:36:49.796580 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.796554    2544 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f421af59-3ffd-49ca-96d4-e580236a60b0-config-file\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:36:49.796580 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:49.796577    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f68gn\" (UniqueName: \"kubernetes.io/projected/f421af59-3ffd-49ca-96d4-e580236a60b0-kube-api-access-f68gn\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:36:50.364096 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:50.364069    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"
Jul 27 19:36:50.364096 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:50.364092    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldwdg" event={"ID":"f421af59-3ffd-49ca-96d4-e580236a60b0","Type":"ContainerDied","Data":"38ac7aa23db0b118071f907fd0fdd3b069909ad8e3f098982252d3b6458c279b"}
Jul 27 19:36:50.364555 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:50.364145    2544 scope.go:117] "RemoveContainer" containerID="670aff8c9c92a4fea992206b93f1f4587f6a9794255a958712ddf66e1905e5e1"
Jul 27 19:36:50.379645 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:50.379618    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:50.381464 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:50.381441    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldwdg"]
Jul 27 19:36:51.840810 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:51.840776    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f421af59-3ffd-49ca-96d4-e580236a60b0" path="/var/lib/kubelet/pods/f421af59-3ffd-49ca-96d4-e580236a60b0/volumes"
Jul 27 19:36:55.046223 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.046181    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"]
Jul 27 19:36:55.046688 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.046673    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f421af59-3ffd-49ca-96d4-e580236a60b0" containerName="limitador"
Jul 27 19:36:55.046752 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.046691    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="f421af59-3ffd-49ca-96d4-e580236a60b0" containerName="limitador"
Jul 27 19:36:55.046848 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.046828    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="f421af59-3ffd-49ca-96d4-e580236a60b0" containerName="limitador"
Jul 27 19:36:55.053248 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.053229    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.055510 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.055491    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 19:36:55.055777 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.055758    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 19:36:55.056754 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.056720    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-szw9g\""
Jul 27 19:36:55.056846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.056739    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 27 19:36:55.059080 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.059061    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"]
Jul 27 19:36:55.141999 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.141972    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlj5n\" (UniqueName: \"kubernetes.io/projected/f6f012ef-7b82-4569-b246-d104fb813b07-kube-api-access-vlj5n\") pod \"postgres-69fb4594cf-jpgsq\" (UID: \"f6f012ef-7b82-4569-b246-d104fb813b07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.142140 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.142011    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6f012ef-7b82-4569-b246-d104fb813b07-data\") pod \"postgres-69fb4594cf-jpgsq\" (UID: \"f6f012ef-7b82-4569-b246-d104fb813b07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.242452 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.242424    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlj5n\" (UniqueName: \"kubernetes.io/projected/f6f012ef-7b82-4569-b246-d104fb813b07-kube-api-access-vlj5n\") pod \"postgres-69fb4594cf-jpgsq\" (UID: \"f6f012ef-7b82-4569-b246-d104fb813b07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.242565 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.242458    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6f012ef-7b82-4569-b246-d104fb813b07-data\") pod \"postgres-69fb4594cf-jpgsq\" (UID: \"f6f012ef-7b82-4569-b246-d104fb813b07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.242873 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.242855    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6f012ef-7b82-4569-b246-d104fb813b07-data\") pod \"postgres-69fb4594cf-jpgsq\" (UID: \"f6f012ef-7b82-4569-b246-d104fb813b07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.250657 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.250638    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlj5n\" (UniqueName: \"kubernetes.io/projected/f6f012ef-7b82-4569-b246-d104fb813b07-kube-api-access-vlj5n\") pod \"postgres-69fb4594cf-jpgsq\" (UID: \"f6f012ef-7b82-4569-b246-d104fb813b07\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.364398 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.364376    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:36:55.494401 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:55.494377    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"]
Jul 27 19:36:55.496817 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:36:55.496788    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f012ef_7b82_4569_b246_d104fb813b07.slice/crio-89ca9ae61504e098675fc494862911cc5ba0def8e59d2658a574865d7026ec2e WatchSource:0}: Error finding container 89ca9ae61504e098675fc494862911cc5ba0def8e59d2658a574865d7026ec2e: Status 404 returned error can't find the container with id 89ca9ae61504e098675fc494862911cc5ba0def8e59d2658a574865d7026ec2e
Jul 27 19:36:56.397309 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:36:56.397270    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq" event={"ID":"f6f012ef-7b82-4569-b246-d104fb813b07","Type":"ContainerStarted","Data":"89ca9ae61504e098675fc494862911cc5ba0def8e59d2658a574865d7026ec2e"}
Jul 27 19:37:01.425443 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:01.425402    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq" event={"ID":"f6f012ef-7b82-4569-b246-d104fb813b07","Type":"ContainerStarted","Data":"5031e9fc956a05378af4dd437e74eaee0eb0849759d1ce8fdb103d14f77c14ec"}
Jul 27 19:37:01.425943 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:01.425562    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:37:01.442820 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:01.442776    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq" podStartSLOduration=1.4261659070000001 podStartE2EDuration="6.4427626s" totalImagesPullingTime="5.016596693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:01.441605085 +0000 UTC m=+640.099314657" watchObservedRunningTime="2026-07-27 19:37:01.4427626 +0000 UTC m=+640.100472128"
Jul 27 19:37:07.457930 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:07.457898    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jpgsq"
Jul 27 19:37:08.221095 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.221064    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85c89b6ccc-f6k6p"]
Jul 27 19:37:08.223457 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.223441    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:08.231259 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.231226    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85c89b6ccc-f6k6p"]
Jul 27 19:37:08.357014 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.356992    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4htg\" (UniqueName: \"kubernetes.io/projected/48fae198-d642-4041-ace1-e36bd905b967-kube-api-access-r4htg\") pod \"authorino-85c89b6ccc-f6k6p\" (UID: \"48fae198-d642-4041-ace1-e36bd905b967\") " pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:08.378846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.378819    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85c89b6ccc-f6k6p"]
Jul 27 19:37:08.379045 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:08.379028    2544 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-r4htg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-85c89b6ccc-f6k6p" podUID="48fae198-d642-4041-ace1-e36bd905b967"
Jul 27 19:37:08.458056 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.458030    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4htg\" (UniqueName: \"kubernetes.io/projected/48fae198-d642-4041-ace1-e36bd905b967-kube-api-access-r4htg\") pod \"authorino-85c89b6ccc-f6k6p\" (UID: \"48fae198-d642-4041-ace1-e36bd905b967\") " pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:08.463682 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.463663    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:08.465658 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.465639    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4htg\" (UniqueName: \"kubernetes.io/projected/48fae198-d642-4041-ace1-e36bd905b967-kube-api-access-r4htg\") pod \"authorino-85c89b6ccc-f6k6p\" (UID: \"48fae198-d642-4041-ace1-e36bd905b967\") " pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:08.476904 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.476848    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:08.558560 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.558533    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4htg\" (UniqueName: \"kubernetes.io/projected/48fae198-d642-4041-ace1-e36bd905b967-kube-api-access-r4htg\") pod \"48fae198-d642-4041-ace1-e36bd905b967\" (UID: \"48fae198-d642-4041-ace1-e36bd905b967\") "
Jul 27 19:37:08.560661 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.560636    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48fae198-d642-4041-ace1-e36bd905b967-kube-api-access-r4htg" (OuterVolumeSpecName: "kube-api-access-r4htg") pod "48fae198-d642-4041-ace1-e36bd905b967" (UID: "48fae198-d642-4041-ace1-e36bd905b967"). InnerVolumeSpecName "kube-api-access-r4htg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:37:08.659662 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:08.659640    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4htg\" (UniqueName: \"kubernetes.io/projected/48fae198-d642-4041-ace1-e36bd905b967-kube-api-access-r4htg\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:37:09.468057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.468030    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85c89b6ccc-f6k6p"
Jul 27 19:37:09.495576 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.495550    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85c89b6ccc-f6k6p"]
Jul 27 19:37:09.500986 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.500966    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85c89b6ccc-f6k6p"]
Jul 27 19:37:09.548007 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.547981    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4w7rw"]
Jul 27 19:37:09.548197 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.548173    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4w7rw" podUID="7808e221-949e-4f47-a12d-b1e71a63857c" containerName="authorino" containerID="cri-o://0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600" gracePeriod=30
Jul 27 19:37:09.787871 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.787847    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:37:09.845689 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.845661    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48fae198-d642-4041-ace1-e36bd905b967" path="/var/lib/kubelet/pods/48fae198-d642-4041-ace1-e36bd905b967/volumes"
Jul 27 19:37:09.867631 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.867610    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v5b8\" (UniqueName: \"kubernetes.io/projected/7808e221-949e-4f47-a12d-b1e71a63857c-kube-api-access-2v5b8\") pod \"7808e221-949e-4f47-a12d-b1e71a63857c\" (UID: \"7808e221-949e-4f47-a12d-b1e71a63857c\") "
Jul 27 19:37:09.869499 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.869471    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7808e221-949e-4f47-a12d-b1e71a63857c-kube-api-access-2v5b8" (OuterVolumeSpecName: "kube-api-access-2v5b8") pod "7808e221-949e-4f47-a12d-b1e71a63857c" (UID: "7808e221-949e-4f47-a12d-b1e71a63857c"). InnerVolumeSpecName "kube-api-access-2v5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:37:09.969061 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:09.969035    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v5b8\" (UniqueName: \"kubernetes.io/projected/7808e221-949e-4f47-a12d-b1e71a63857c-kube-api-access-2v5b8\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:37:10.473864 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.473836    2544 generic.go:356] "Generic (PLEG): container finished" podID="7808e221-949e-4f47-a12d-b1e71a63857c" containerID="0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600" exitCode=0
Jul 27 19:37:10.474290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.473903    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4w7rw"
Jul 27 19:37:10.474290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.473911    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4w7rw" event={"ID":"7808e221-949e-4f47-a12d-b1e71a63857c","Type":"ContainerDied","Data":"0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600"}
Jul 27 19:37:10.474290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.473951    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4w7rw" event={"ID":"7808e221-949e-4f47-a12d-b1e71a63857c","Type":"ContainerDied","Data":"09c3a373e6e54c4040a4e40105b6771d241afc9d93100ab31ff3884ce84580ad"}
Jul 27 19:37:10.474290 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.473970    2544 scope.go:117] "RemoveContainer" containerID="0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600"
Jul 27 19:37:10.482605 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.482588    2544 scope.go:117] "RemoveContainer" containerID="0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600"
Jul 27 19:37:10.482867 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:10.482849    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600\": container with ID starting with 0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600 not found: ID does not exist" containerID="0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600"
Jul 27 19:37:10.482926 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.482877    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600"} err="failed to get container status \"0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600\": rpc error: code = NotFound desc = could not find container \"0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600\": container with ID starting with 0b98563552f505755517203c8ce69374f88e2c6784f56c0ecf29b4a7da6a4600 not found: ID does not exist"
Jul 27 19:37:10.493367 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.493348    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4w7rw"]
Jul 27 19:37:10.496787 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:10.496767    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4w7rw"]
Jul 27 19:37:11.840714 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:11.840683    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7808e221-949e-4f47-a12d-b1e71a63857c" path="/var/lib/kubelet/pods/7808e221-949e-4f47-a12d-b1e71a63857c/volumes"
Jul 27 19:37:14.031545 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.031516    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c5d7ff667-p9d6l"]
Jul 27 19:37:14.031908 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.031886    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7808e221-949e-4f47-a12d-b1e71a63857c" containerName="authorino"
Jul 27 19:37:14.031908 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.031899    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="7808e221-949e-4f47-a12d-b1e71a63857c" containerName="authorino"
Jul 27 19:37:14.031978 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.031951    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="7808e221-949e-4f47-a12d-b1e71a63857c" containerName="authorino"
Jul 27 19:37:14.034702 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.034681    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.037109 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.037087    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 19:37:14.037240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.037143    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7sbpd\""
Jul 27 19:37:14.037240 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.037149    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 19:37:14.053578 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.053546    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c5d7ff667-p9d6l"]
Jul 27 19:37:14.200634 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.200598    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lkwf\" (UniqueName: \"kubernetes.io/projected/c16e2d12-997b-4c1e-b02e-5bfd36044b76-kube-api-access-5lkwf\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.200812 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.200664    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.231014 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.230988    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-686ff9dc49-nqwcp"]
Jul 27 19:37:14.233339 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.233322    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:14.243579 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.243553    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-nqwcp"]
Jul 27 19:37:14.301578 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.301519    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lkwf\" (UniqueName: \"kubernetes.io/projected/c16e2d12-997b-4c1e-b02e-5bfd36044b76-kube-api-access-5lkwf\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.301578 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.301561    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.301759 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:14.301656    2544 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 27 19:37:14.301759 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:14.301708    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert podName:c16e2d12-997b-4c1e-b02e-5bfd36044b76 nodeName:}" failed. No retries permitted until 2026-07-27 19:37:14.801694856 +0000 UTC m=+653.459404366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert") pod "maas-controller-7c5d7ff667-p9d6l" (UID: "c16e2d12-997b-4c1e-b02e-5bfd36044b76") : secret "maas-controller-webhook-cert" not found
Jul 27 19:37:14.309971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.309950    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lkwf\" (UniqueName: \"kubernetes.io/projected/c16e2d12-997b-4c1e-b02e-5bfd36044b76-kube-api-access-5lkwf\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.402748 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.402707    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:14.402866 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.402842    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qlwk\" (UniqueName: \"kubernetes.io/projected/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-kube-api-access-5qlwk\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:14.503273 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.503250    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qlwk\" (UniqueName: \"kubernetes.io/projected/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-kube-api-access-5qlwk\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:14.503365 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.503291    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:14.503417 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:14.503406    2544 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 27 19:37:14.503465 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:14.503457    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert podName:8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04 nodeName:}" failed. No retries permitted until 2026-07-27 19:37:15.003445634 +0000 UTC m=+653.661155141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert") pod "maas-controller-686ff9dc49-nqwcp" (UID: "8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04") : secret "maas-controller-webhook-cert" not found
Jul 27 19:37:14.511129 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.511109    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qlwk\" (UniqueName: \"kubernetes.io/projected/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-kube-api-access-5qlwk\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:14.805362 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:14.805337    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:14.805472 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:14.805449    2544 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 27 19:37:14.805510 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:14.805503    2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert podName:c16e2d12-997b-4c1e-b02e-5bfd36044b76 nodeName:}" failed. No retries permitted until 2026-07-27 19:37:15.805490098 +0000 UTC m=+654.463199605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert") pod "maas-controller-7c5d7ff667-p9d6l" (UID: "c16e2d12-997b-4c1e-b02e-5bfd36044b76") : secret "maas-controller-webhook-cert" not found
Jul 27 19:37:15.007305 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.007279    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:15.009597 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.009567    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert\") pod \"maas-controller-686ff9dc49-nqwcp\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") " pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:15.144283 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.144253    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:15.276780 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.276751    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-nqwcp"]
Jul 27 19:37:15.279283 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:37:15.279257    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe4c3a7_eb69_4145_ac67_ad33e3f1ac04.slice/crio-9a980b6b59c708f5726734c615fd07aa67bd8d09552fe4ef5db6fdb3717c4ff4 WatchSource:0}: Error finding container 9a980b6b59c708f5726734c615fd07aa67bd8d09552fe4ef5db6fdb3717c4ff4: Status 404 returned error can't find the container with id 9a980b6b59c708f5726734c615fd07aa67bd8d09552fe4ef5db6fdb3717c4ff4
Jul 27 19:37:15.498570 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.498493    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-nqwcp" event={"ID":"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04","Type":"ContainerStarted","Data":"9a980b6b59c708f5726734c615fd07aa67bd8d09552fe4ef5db6fdb3717c4ff4"}
Jul 27 19:37:15.815268 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.815193    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:15.817439 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.817422    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") pod \"maas-controller-7c5d7ff667-p9d6l\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") " pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:15.844562 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.844539    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:15.971249 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:15.971219    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c5d7ff667-p9d6l"]
Jul 27 19:37:15.973948 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:37:15.973922    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc16e2d12_997b_4c1e_b02e_5bfd36044b76.slice/crio-497211dc31e79abb412fc4c3366c6cc30ffe54ef40ab1c64dd99361103d4fd65 WatchSource:0}: Error finding container 497211dc31e79abb412fc4c3366c6cc30ffe54ef40ab1c64dd99361103d4fd65: Status 404 returned error can't find the container with id 497211dc31e79abb412fc4c3366c6cc30ffe54ef40ab1c64dd99361103d4fd65
Jul 27 19:37:16.505200 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:16.505149    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l" event={"ID":"c16e2d12-997b-4c1e-b02e-5bfd36044b76","Type":"ContainerStarted","Data":"497211dc31e79abb412fc4c3366c6cc30ffe54ef40ab1c64dd99361103d4fd65"}
Jul 27 19:37:18.516929 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:18.516890    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-nqwcp" event={"ID":"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04","Type":"ContainerStarted","Data":"0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd"}
Jul 27 19:37:18.517381 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:18.516981    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:18.518517 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:18.518489    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l" event={"ID":"c16e2d12-997b-4c1e-b02e-5bfd36044b76","Type":"ContainerStarted","Data":"d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5"}
Jul 27 19:37:18.518629 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:18.518570    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:18.549839 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:18.549796    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-686ff9dc49-nqwcp" podStartSLOduration=1.722537126 podStartE2EDuration="4.549782749s" totalImagesPullingTime="2.827245623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:18.548707362 +0000 UTC m=+657.206416892" watchObservedRunningTime="2026-07-27 19:37:18.549782749 +0000 UTC m=+657.207492273"
Jul 27 19:37:18.585513 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:18.585471    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l" podStartSLOduration=2.446389502 podStartE2EDuration="4.585457691s" totalImagesPullingTime="2.139068189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:18.583823601 +0000 UTC m=+657.241533142" watchObservedRunningTime="2026-07-27 19:37:18.585457691 +0000 UTC m=+657.243167220"
Jul 27 19:37:29.525515 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.525484    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:29.525951 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.525555    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:37:29.592979 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.592946    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c5d7ff667-p9d6l"]
Jul 27 19:37:29.593165 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.593144    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l" podUID="c16e2d12-997b-4c1e-b02e-5bfd36044b76" containerName="manager" containerID="cri-o://d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5" gracePeriod=30
Jul 27 19:37:29.849540 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.849513    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:29.932391 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.932365    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lkwf\" (UniqueName: \"kubernetes.io/projected/c16e2d12-997b-4c1e-b02e-5bfd36044b76-kube-api-access-5lkwf\") pod \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") "
Jul 27 19:37:29.932555 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.932409    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") pod \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\" (UID: \"c16e2d12-997b-4c1e-b02e-5bfd36044b76\") "
Jul 27 19:37:29.934664 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.934640    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c16e2d12-997b-4c1e-b02e-5bfd36044b76-kube-api-access-5lkwf" (OuterVolumeSpecName: "kube-api-access-5lkwf") pod "c16e2d12-997b-4c1e-b02e-5bfd36044b76" (UID: "c16e2d12-997b-4c1e-b02e-5bfd36044b76"). InnerVolumeSpecName "kube-api-access-5lkwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:37:29.934664 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:29.934642    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c16e2d12-997b-4c1e-b02e-5bfd36044b76" (UID: "c16e2d12-997b-4c1e-b02e-5bfd36044b76"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:37:30.033847 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.033823    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lkwf\" (UniqueName: \"kubernetes.io/projected/c16e2d12-997b-4c1e-b02e-5bfd36044b76-kube-api-access-5lkwf\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:37:30.033847 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.033843    2544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c16e2d12-997b-4c1e-b02e-5bfd36044b76-webhook-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:37:30.575663 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.575631    2544 generic.go:356] "Generic (PLEG): container finished" podID="c16e2d12-997b-4c1e-b02e-5bfd36044b76" containerID="d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5" exitCode=0
Jul 27 19:37:30.576130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.575669    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l"
Jul 27 19:37:30.576130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.575717    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l" event={"ID":"c16e2d12-997b-4c1e-b02e-5bfd36044b76","Type":"ContainerDied","Data":"d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5"}
Jul 27 19:37:30.576130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.575790    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c5d7ff667-p9d6l" event={"ID":"c16e2d12-997b-4c1e-b02e-5bfd36044b76","Type":"ContainerDied","Data":"497211dc31e79abb412fc4c3366c6cc30ffe54ef40ab1c64dd99361103d4fd65"}
Jul 27 19:37:30.576130 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.575825    2544 scope.go:117] "RemoveContainer" containerID="d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5"
Jul 27 19:37:30.591091 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.591074    2544 scope.go:117] "RemoveContainer" containerID="d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5"
Jul 27 19:37:30.591338 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:37:30.591318    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5\": container with ID starting with d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5 not found: ID does not exist" containerID="d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5"
Jul 27 19:37:30.591405 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.591350    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5"} err="failed to get container status \"d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5\": rpc error: code = NotFound desc = could not find container \"d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5\": container with ID starting with d824fbcf75d3bdd1e56a6b5fdf1f8ba8a6fd939ba1758749669313fc169335a5 not found: ID does not exist"
Jul 27 19:37:30.600523 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.600502    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c5d7ff667-p9d6l"]
Jul 27 19:37:30.603815 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:30.603793    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c5d7ff667-p9d6l"]
Jul 27 19:37:31.841164 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:31.841133    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c16e2d12-997b-4c1e-b02e-5bfd36044b76" path="/var/lib/kubelet/pods/c16e2d12-997b-4c1e-b02e-5bfd36044b76/volumes"
Jul 27 19:37:51.675971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.675936    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"]
Jul 27 19:37:51.676433 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.676340    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c16e2d12-997b-4c1e-b02e-5bfd36044b76" containerName="manager"
Jul 27 19:37:51.676433 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.676360    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="c16e2d12-997b-4c1e-b02e-5bfd36044b76" containerName="manager"
Jul 27 19:37:51.676433 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.676419    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="c16e2d12-997b-4c1e-b02e-5bfd36044b76" containerName="manager"
Jul 27 19:37:51.683043 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.683024    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.686544 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.686523    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-86dhg\""
Jul 27 19:37:51.686687 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.686558    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 27 19:37:51.690567 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.690543    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"]
Jul 27 19:37:51.797404 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.797377    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-openshift-service-ca\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.797555 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.797412    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn4m9\" (UniqueName: \"kubernetes.io/projected/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-kube-api-access-nn4m9\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.797555 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.797482    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-maas-api-tls\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.897923 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.897894    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-openshift-service-ca\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.898097 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.897965    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn4m9\" (UniqueName: \"kubernetes.io/projected/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-kube-api-access-nn4m9\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.898097 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.898035    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-maas-api-tls\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.898596 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.898571    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-openshift-service-ca\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.900714 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.900693    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-maas-api-tls\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.906012 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.905994    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn4m9\" (UniqueName: \"kubernetes.io/projected/ccf9bee0-6077-4dc7-9b60-16bbaf329e3b-kube-api-access-nn4m9\") pod \"maas-api-59d6587f6d-qwh5v\" (UID: \"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b\") " pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:51.994906 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:51.994832    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:52.133325 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.133231    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"]
Jul 27 19:37:52.135813 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:37:52.135781    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf9bee0_6077_4dc7_9b60_16bbaf329e3b.slice/crio-e4a9049e641c966e79b04c72e740d7732f8b90f8a363907f68d5158640dfc912 WatchSource:0}: Error finding container e4a9049e641c966e79b04c72e740d7732f8b90f8a363907f68d5158640dfc912: Status 404 returned error can't find the container with id e4a9049e641c966e79b04c72e740d7732f8b90f8a363907f68d5158640dfc912
Jul 27 19:37:52.143377 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.143359    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"]
Jul 27 19:37:52.148513 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.148496    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.149472 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.149448    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-gbhk5"]
Jul 27 19:37:52.151036 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.151015    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sd8ff\""
Jul 27 19:37:52.151135 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.151015    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 27 19:37:52.152923 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.152906    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.157818 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.157090    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"]
Jul 27 19:37:52.161767 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.161746    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-gbhk5"]
Jul 27 19:37:52.200035 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.200006    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602f0fdc-6d53-45ad-a048-112e95eb5f7b-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-gbhk5\" (UID: \"602f0fdc-6d53-45ad-a048-112e95eb5f7b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.200144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.200063    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47gqs\" (UniqueName: \"kubernetes.io/projected/3faa46e7-6a90-42b3-89c0-253e7ed878d6-kube-api-access-47gqs\") pod \"payload-pre-processing-687dbb6cf6-ct46q\" (UID: \"3faa46e7-6a90-42b3-89c0-253e7ed878d6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.200144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.200091    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5rt5\" (UniqueName: \"kubernetes.io/projected/602f0fdc-6d53-45ad-a048-112e95eb5f7b-kube-api-access-c5rt5\") pod \"payload-processing-5cc6cd444b-gbhk5\" (UID: \"602f0fdc-6d53-45ad-a048-112e95eb5f7b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.200144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.200123    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faa46e7-6a90-42b3-89c0-253e7ed878d6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-ct46q\" (UID: \"3faa46e7-6a90-42b3-89c0-253e7ed878d6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.300841 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.300724    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602f0fdc-6d53-45ad-a048-112e95eb5f7b-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-gbhk5\" (UID: \"602f0fdc-6d53-45ad-a048-112e95eb5f7b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.300989 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.300856    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47gqs\" (UniqueName: \"kubernetes.io/projected/3faa46e7-6a90-42b3-89c0-253e7ed878d6-kube-api-access-47gqs\") pod \"payload-pre-processing-687dbb6cf6-ct46q\" (UID: \"3faa46e7-6a90-42b3-89c0-253e7ed878d6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.300989 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.300896    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5rt5\" (UniqueName: \"kubernetes.io/projected/602f0fdc-6d53-45ad-a048-112e95eb5f7b-kube-api-access-c5rt5\") pod \"payload-processing-5cc6cd444b-gbhk5\" (UID: \"602f0fdc-6d53-45ad-a048-112e95eb5f7b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.300989 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.300942    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faa46e7-6a90-42b3-89c0-253e7ed878d6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-ct46q\" (UID: \"3faa46e7-6a90-42b3-89c0-253e7ed878d6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.301469 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.301448    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/602f0fdc-6d53-45ad-a048-112e95eb5f7b-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-gbhk5\" (UID: \"602f0fdc-6d53-45ad-a048-112e95eb5f7b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.301630 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.301611    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3faa46e7-6a90-42b3-89c0-253e7ed878d6-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-ct46q\" (UID: \"3faa46e7-6a90-42b3-89c0-253e7ed878d6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.308449 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.308416    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5rt5\" (UniqueName: \"kubernetes.io/projected/602f0fdc-6d53-45ad-a048-112e95eb5f7b-kube-api-access-c5rt5\") pod \"payload-processing-5cc6cd444b-gbhk5\" (UID: \"602f0fdc-6d53-45ad-a048-112e95eb5f7b\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.308570 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.308473    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47gqs\" (UniqueName: \"kubernetes.io/projected/3faa46e7-6a90-42b3-89c0-253e7ed878d6-kube-api-access-47gqs\") pod \"payload-pre-processing-687dbb6cf6-ct46q\" (UID: \"3faa46e7-6a90-42b3-89c0-253e7ed878d6\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.462536 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.462499    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:52.470834 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.470817    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:52.608152 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.608124    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"]
Jul 27 19:37:52.611475 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:37:52.611448    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3faa46e7_6a90_42b3_89c0_253e7ed878d6.slice/crio-0dabb7b6ef73a2aa91714e4de2e81d507b7adb66cf9b1af77accd3991e72a177 WatchSource:0}: Error finding container 0dabb7b6ef73a2aa91714e4de2e81d507b7adb66cf9b1af77accd3991e72a177: Status 404 returned error can't find the container with id 0dabb7b6ef73a2aa91714e4de2e81d507b7adb66cf9b1af77accd3991e72a177
Jul 27 19:37:52.650365 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.650342    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-gbhk5"]
Jul 27 19:37:52.652154 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:37:52.652117    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602f0fdc_6d53_45ad_a048_112e95eb5f7b.slice/crio-b56450de4150771779cb66cb3403b6bc797dffbc2cf6dbe1750f9aabaacf3a5c WatchSource:0}: Error finding container b56450de4150771779cb66cb3403b6bc797dffbc2cf6dbe1750f9aabaacf3a5c: Status 404 returned error can't find the container with id b56450de4150771779cb66cb3403b6bc797dffbc2cf6dbe1750f9aabaacf3a5c
Jul 27 19:37:52.683712 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.683668    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q" event={"ID":"3faa46e7-6a90-42b3-89c0-253e7ed878d6","Type":"ContainerStarted","Data":"0dabb7b6ef73a2aa91714e4de2e81d507b7adb66cf9b1af77accd3991e72a177"}
Jul 27 19:37:52.685221 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.685189    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v" event={"ID":"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b","Type":"ContainerStarted","Data":"e4a9049e641c966e79b04c72e740d7732f8b90f8a363907f68d5158640dfc912"}
Jul 27 19:37:52.686522 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:52.686495    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5" event={"ID":"602f0fdc-6d53-45ad-a048-112e95eb5f7b","Type":"ContainerStarted","Data":"b56450de4150771779cb66cb3403b6bc797dffbc2cf6dbe1750f9aabaacf3a5c"}
Jul 27 19:37:56.715950 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.715913    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q" event={"ID":"3faa46e7-6a90-42b3-89c0-253e7ed878d6","Type":"ContainerStarted","Data":"aea2534a197925755bbf3fa0b544701af70ce3e2f23e3e52f3fdfe7effd337ac"}
Jul 27 19:37:56.716405 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.716080    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:37:56.717563 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.717536    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v" event={"ID":"ccf9bee0-6077-4dc7-9b60-16bbaf329e3b","Type":"ContainerStarted","Data":"f80c31e759b572ec8e9ee7846bf10333c9a9e6882cdf6621192b164f97c69917"}
Jul 27 19:37:56.717705 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.717660    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:37:56.718983 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.718963    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5" event={"ID":"602f0fdc-6d53-45ad-a048-112e95eb5f7b","Type":"ContainerStarted","Data":"32686f656c0f29ffa8c5aea6af3efd588b3e9fbcdbf18798de70e0712e81da66"}
Jul 27 19:37:56.719073 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.719007    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:37:56.733854 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.733816    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q" podStartSLOduration=1.309440193 podStartE2EDuration="4.733804008s" totalImagesPullingTime="3.424363815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:56.731873159 +0000 UTC m=+695.389582687" watchObservedRunningTime="2026-07-27 19:37:56.733804008 +0000 UTC m=+695.391513537"
Jul 27 19:37:56.750681 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.750643    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v" podStartSLOduration=1.9335915510000001 podStartE2EDuration="5.75062975s" totalImagesPullingTime="3.817038199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:56.749264857 +0000 UTC m=+695.406974387" watchObservedRunningTime="2026-07-27 19:37:56.75062975 +0000 UTC m=+695.408339279"
Jul 27 19:37:56.765523 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:56.765479    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5" podStartSLOduration=1.374472976 podStartE2EDuration="4.765468068s" totalImagesPullingTime="3.390995092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:37:56.764816463 +0000 UTC m=+695.422525993" watchObservedRunningTime="2026-07-27 19:37:56.765468068 +0000 UTC m=+695.423177597"
Jul 27 19:37:59.734234 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:59.734178    2544 generic.go:356] "Generic (PLEG): container finished" podID="b8a643f3-33b1-4b82-8021-0f632ef6a3a2" containerID="227accfc5b54022f6802e3fad3d8c5871c70273150a6e95546528bc33611af15" exitCode=0
Jul 27 19:37:59.734662 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:59.734255    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" event={"ID":"b8a643f3-33b1-4b82-8021-0f632ef6a3a2","Type":"ContainerDied","Data":"227accfc5b54022f6802e3fad3d8c5871c70273150a6e95546528bc33611af15"}
Jul 27 19:37:59.734724 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:59.734709    2544 scope.go:117] "RemoveContainer" containerID="227accfc5b54022f6802e3fad3d8c5871c70273150a6e95546528bc33611af15"
Jul 27 19:37:59.735720 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:59.735692    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:37:59.735857 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:59.735774    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:37:59.735857 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:37:59.735809    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:38:00.741068 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:00.741029    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" event={"ID":"b8a643f3-33b1-4b82-8021-0f632ef6a3a2","Type":"ContainerStarted","Data":"44955c6aab67ae6e0b57485f1fd348cb7bf1d0fc3ab9b7944cee18551d369b5a"}
Jul 27 19:38:01.721023 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:01.720991    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:38:01.722244 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:01.722215    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-gxwfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 27 19:38:01.722366 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:01.722263    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" podUID="b8a643f3-33b1-4b82-8021-0f632ef6a3a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 27 19:38:02.721598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:02.721563    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-gxwfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 27 19:38:02.721988 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:02.721621    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" podUID="b8a643f3-33b1-4b82-8021-0f632ef6a3a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 27 19:38:02.729788 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:02.729764    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59d6587f6d-qwh5v"
Jul 27 19:38:03.269650 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:03.269613    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:38:03.721956 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:03.721923    2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-gxwfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 27 19:38:03.722341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:03.721976    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr" podUID="b8a643f3-33b1-4b82-8021-0f632ef6a3a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 27 19:38:04.724916 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:04.724884    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:38:04.725775 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:04.725757    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-gxwfr"
Jul 27 19:38:07.725941 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:07.725911    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-ct46q"
Jul 27 19:38:07.726361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:07.726086    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-gbhk5"
Jul 27 19:38:30.325054 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.325020    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"]
Jul 27 19:38:30.333852 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.333824    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.336477 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.336451    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 19:38:30.336648 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.336625    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 27 19:38:30.337658 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.337635    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 19:38:30.337658 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.337649    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zc62b\""
Jul 27 19:38:30.338046 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.338024    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"]
Jul 27 19:38:30.439582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.439546    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5qvj\" (UniqueName: \"kubernetes.io/projected/517465b2-220d-4d59-8642-ba177f09a704-kube-api-access-w5qvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.439777 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.439588    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.439777 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.439623    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.439777 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.439648    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.439777 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.439679    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/517465b2-220d-4d59-8642-ba177f09a704-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.439922 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.439782    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541014 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.540969    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/517465b2-220d-4d59-8642-ba177f09a704-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541203 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541042    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541203 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541088    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5qvj\" (UniqueName: \"kubernetes.io/projected/517465b2-220d-4d59-8642-ba177f09a704-kube-api-access-w5qvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541203 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541126    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541203 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541153    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541203 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541182    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541450 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541431    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541507 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541488    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.541565 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.541509    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.543342 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.543322    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/517465b2-220d-4d59-8642-ba177f09a704-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.543579 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.543562    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/517465b2-220d-4d59-8642-ba177f09a704-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.548434 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.548416    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5qvj\" (UniqueName: \"kubernetes.io/projected/517465b2-220d-4d59-8642-ba177f09a704-kube-api-access-w5qvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm\" (UID: \"517465b2-220d-4d59-8642-ba177f09a704\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.645870 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.645838    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:30.778579 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.778513    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"]
Jul 27 19:38:30.781247 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:38:30.781206    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517465b2_220d_4d59_8642_ba177f09a704.slice/crio-e4ab43ce33c3b9230b5bd552d8f332a8547a490415d3066c43925f649cf908ad WatchSource:0}: Error finding container e4ab43ce33c3b9230b5bd552d8f332a8547a490415d3066c43925f649cf908ad: Status 404 returned error can't find the container with id e4ab43ce33c3b9230b5bd552d8f332a8547a490415d3066c43925f649cf908ad
Jul 27 19:38:30.895257 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:30.895222    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm" event={"ID":"517465b2-220d-4d59-8642-ba177f09a704","Type":"ContainerStarted","Data":"e4ab43ce33c3b9230b5bd552d8f332a8547a490415d3066c43925f649cf908ad"}
Jul 27 19:38:31.902156 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:31.902117    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm" event={"ID":"517465b2-220d-4d59-8642-ba177f09a704","Type":"ContainerStarted","Data":"404c99095fc14225043c9cc946a02b674f838b19f8f39cc8ea6625b75d8d007a"}
Jul 27 19:38:31.922508 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:31.922426    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm" podStartSLOduration=0.954443742 podStartE2EDuration="1.922409174s" totalImagesPullingTime="967.965432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:38:31.920381191 +0000 UTC m=+730.578090720" watchObservedRunningTime="2026-07-27 19:38:31.922409174 +0000 UTC m=+730.580118702"
Jul 27 19:38:38.324569 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.324536    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"]
Jul 27 19:38:38.328453 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.328429    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.330852 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.330833    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 27 19:38:38.337248 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.337223    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"]
Jul 27 19:38:38.510964 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.510932    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.511141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.510982    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22cdk\" (UniqueName: \"kubernetes.io/projected/b8638429-e9da-4447-bac3-fd6393a74fa9-kube-api-access-22cdk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.511141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.511056    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.511141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.511097    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8638429-e9da-4447-bac3-fd6393a74fa9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.511141 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.511117    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.511288 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.511144    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.612582 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.612550    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.612783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.612601    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8638429-e9da-4447-bac3-fd6393a74fa9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.612783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.612621    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.612783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.612643    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.612783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.612674    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.612783 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.612710    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22cdk\" (UniqueName: \"kubernetes.io/projected/b8638429-e9da-4447-bac3-fd6393a74fa9-kube-api-access-22cdk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.613122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.613088    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.613122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.613116    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.613265 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.613169    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.614917 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.614896    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8638429-e9da-4447-bac3-fd6393a74fa9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.615089 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.615071    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8638429-e9da-4447-bac3-fd6393a74fa9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.620388 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.620362    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22cdk\" (UniqueName: \"kubernetes.io/projected/b8638429-e9da-4447-bac3-fd6393a74fa9-kube-api-access-22cdk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6z7zc\" (UID: \"b8638429-e9da-4447-bac3-fd6393a74fa9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.639619 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.639584    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:38.781375 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.781336    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"]
Jul 27 19:38:38.783868 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:38:38.783836    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8638429_e9da_4447_bac3_fd6393a74fa9.slice/crio-a68480b3684faeb1ec08e21ca5a81d9dbd3dfe0a96c1a3f5aa94663650e5bd70 WatchSource:0}: Error finding container a68480b3684faeb1ec08e21ca5a81d9dbd3dfe0a96c1a3f5aa94663650e5bd70: Status 404 returned error can't find the container with id a68480b3684faeb1ec08e21ca5a81d9dbd3dfe0a96c1a3f5aa94663650e5bd70
Jul 27 19:38:38.937899 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:38.937815    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc" event={"ID":"b8638429-e9da-4447-bac3-fd6393a74fa9","Type":"ContainerStarted","Data":"a68480b3684faeb1ec08e21ca5a81d9dbd3dfe0a96c1a3f5aa94663650e5bd70"}
Jul 27 19:38:39.944291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:39.944260    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc" event={"ID":"b8638429-e9da-4447-bac3-fd6393a74fa9","Type":"ContainerStarted","Data":"6f4ba95f0ce88fb09dedbd581fac6d8719083671e931ea7025085c74b9a2191c"}
Jul 27 19:38:39.961970 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:39.961916    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc" podStartSLOduration=1.677720769 podStartE2EDuration="1.961898514s" totalImagesPullingTime="284.177745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:38:39.961143631 +0000 UTC m=+738.618853161" watchObservedRunningTime="2026-07-27 19:38:39.961898514 +0000 UTC m=+738.619608043"
Jul 27 19:38:40.646637 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:40.646593    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:40.646872 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:40.646658    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:40.659274 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:40.659251    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:40.959908 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:40.959820    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm"
Jul 27 19:38:48.640265 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:48.640215    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:48.640803 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:48.640299    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:48.652858 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:48.652831    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:38:49.000679 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:38:49.000588    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6z7zc"
Jul 27 19:39:07.225067 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.225034    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"]
Jul 27 19:39:07.230773 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.230746    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.233597 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.233568    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 27 19:39:07.237408 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.237384    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"]
Jul 27 19:39:07.275911 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.275877    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.275911 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.275913    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4hc2\" (UniqueName: \"kubernetes.io/projected/4098324b-2360-401f-9684-7ba7ab404f24-kube-api-access-w4hc2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.276122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.275951    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.276122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.276003    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.276122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.276031    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4098324b-2360-401f-9684-7ba7ab404f24-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.276122 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.276091    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.376936 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.376894    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.376936 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.376935    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4hc2\" (UniqueName: \"kubernetes.io/projected/4098324b-2360-401f-9684-7ba7ab404f24-kube-api-access-w4hc2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377160 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.376963    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377160 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.376983    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377160 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.376998    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4098324b-2360-401f-9684-7ba7ab404f24-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377160 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.377028    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377306 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.377282    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377401 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.377378    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.377464 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.377396    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.379215 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.379189    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4098324b-2360-401f-9684-7ba7ab404f24-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.379444 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.379427    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4098324b-2360-401f-9684-7ba7ab404f24-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.384480 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.384455    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4hc2\" (UniqueName: \"kubernetes.io/projected/4098324b-2360-401f-9684-7ba7ab404f24-kube-api-access-w4hc2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz\" (UID: \"4098324b-2360-401f-9684-7ba7ab404f24\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.541812 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.541672    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:07.680470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:07.680441    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"]
Jul 27 19:39:07.682936 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:39:07.682798    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4098324b_2360_401f_9684_7ba7ab404f24.slice/crio-5fec9bcab583f41d766e86d327e9e628fef5538d36da4bcfbe4f758947c397f3 WatchSource:0}: Error finding container 5fec9bcab583f41d766e86d327e9e628fef5538d36da4bcfbe4f758947c397f3: Status 404 returned error can't find the container with id 5fec9bcab583f41d766e86d327e9e628fef5538d36da4bcfbe4f758947c397f3
Jul 27 19:39:08.088497 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:08.088459    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz" event={"ID":"4098324b-2360-401f-9684-7ba7ab404f24","Type":"ContainerStarted","Data":"fcf011a7b0dbc08e4cadc2a04c2630eb857c8243bc6296d1bfef4a46122dd22f"}
Jul 27 19:39:08.088677 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:08.088507    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz" event={"ID":"4098324b-2360-401f-9684-7ba7ab404f24","Type":"ContainerStarted","Data":"5fec9bcab583f41d766e86d327e9e628fef5538d36da4bcfbe4f758947c397f3"}
Jul 27 19:39:08.107149 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:08.107031    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz" podStartSLOduration=0.942007347 podStartE2EDuration="1.1070124s" totalImagesPullingTime="165.005053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:39:08.104991055 +0000 UTC m=+766.762700584" watchObservedRunningTime="2026-07-27 19:39:08.1070124 +0000 UTC m=+766.764721928"
Jul 27 19:39:17.542609 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:17.542573    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:17.542609 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:17.542627    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:17.554639 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:17.554605    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:39:18.148246 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:39:18.148218    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz"
Jul 27 19:40:51.281284 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.281247    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-nqwcp"]
Jul 27 19:40:51.281859 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.281543    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-686ff9dc49-nqwcp" podUID="8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" containerName="manager" containerID="cri-o://0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd" gracePeriod=30
Jul 27 19:40:51.540236 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.540180    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:40:51.556509 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.556483    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert\") pod \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") "
Jul 27 19:40:51.556647 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.556525    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qlwk\" (UniqueName: \"kubernetes.io/projected/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-kube-api-access-5qlwk\") pod \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\" (UID: \"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04\") "
Jul 27 19:40:51.559128 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.559096    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" (UID: "8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:40:51.559961 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.559912    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-kube-api-access-5qlwk" (OuterVolumeSpecName: "kube-api-access-5qlwk") pod "8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" (UID: "8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04"). InnerVolumeSpecName "kube-api-access-5qlwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:40:51.603463 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.603421    2544 generic.go:356] "Generic (PLEG): container finished" podID="8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" containerID="0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd" exitCode=0
Jul 27 19:40:51.603648 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.603623    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-nqwcp" event={"ID":"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04","Type":"ContainerDied","Data":"0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd"}
Jul 27 19:40:51.603725 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.603662    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-nqwcp"
Jul 27 19:40:51.603725 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.603711    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-nqwcp" event={"ID":"8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04","Type":"ContainerDied","Data":"9a980b6b59c708f5726734c615fd07aa67bd8d09552fe4ef5db6fdb3717c4ff4"}
Jul 27 19:40:51.604603 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.603763    2544 scope.go:117] "RemoveContainer" containerID="0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd"
Jul 27 19:40:51.613953 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.613937    2544 scope.go:117] "RemoveContainer" containerID="0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd"
Jul 27 19:40:51.614204 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:40:51.614189    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd\": container with ID starting with 0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd not found: ID does not exist" containerID="0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd"
Jul 27 19:40:51.614242 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.614212    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd"} err="failed to get container status \"0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd\": rpc error: code = NotFound desc = could not find container \"0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd\": container with ID starting with 0729c2554d6d728befdee4d79da13084017da0a255a36146509e3e9b4891facd not found: ID does not exist"
Jul 27 19:40:51.625065 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.625043    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-nqwcp"]
Jul 27 19:40:51.628394 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.628367    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-nqwcp"]
Jul 27 19:40:51.657572 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.657550    2544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-webhook-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:40:51.657680 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.657574    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qlwk\" (UniqueName: \"kubernetes.io/projected/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04-kube-api-access-5qlwk\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:40:51.840910 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:40:51.840835    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" path="/var/lib/kubelet/pods/8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04/volumes"
Jul 27 19:41:21.756358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:21.756272    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:41:21.756358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:21.756318    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:41:52.430548 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.430515    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-686ff9dc49-zrnd9"]
Jul 27 19:41:52.431184 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.431139    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" containerName="manager"
Jul 27 19:41:52.431265 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.431189    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" containerName="manager"
Jul 27 19:41:52.431320 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.431311    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fe4c3a7-eb69-4145-ac67-ad33e3f1ac04" containerName="manager"
Jul 27 19:41:52.434429 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.434406    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.436967 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.436944    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 19:41:52.437846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.437829    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7sbpd\""
Jul 27 19:41:52.437935 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.437861    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 19:41:52.443490 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.443471    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-zrnd9"]
Jul 27 19:41:52.494918 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.494883    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef961822-0c5d-4af5-9ad6-479b3b414144-webhook-cert\") pod \"maas-controller-686ff9dc49-zrnd9\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") " pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.495072 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.494949    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2t94\" (UniqueName: \"kubernetes.io/projected/ef961822-0c5d-4af5-9ad6-479b3b414144-kube-api-access-r2t94\") pod \"maas-controller-686ff9dc49-zrnd9\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") " pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.596189 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.596154    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef961822-0c5d-4af5-9ad6-479b3b414144-webhook-cert\") pod \"maas-controller-686ff9dc49-zrnd9\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") " pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.596359 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.596229    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2t94\" (UniqueName: \"kubernetes.io/projected/ef961822-0c5d-4af5-9ad6-479b3b414144-kube-api-access-r2t94\") pod \"maas-controller-686ff9dc49-zrnd9\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") " pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.598616 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.598587    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef961822-0c5d-4af5-9ad6-479b3b414144-webhook-cert\") pod \"maas-controller-686ff9dc49-zrnd9\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") " pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.604223 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.604192    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2t94\" (UniqueName: \"kubernetes.io/projected/ef961822-0c5d-4af5-9ad6-479b3b414144-kube-api-access-r2t94\") pod \"maas-controller-686ff9dc49-zrnd9\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") " pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.745687 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.745604    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:52.885078 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.885014    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-zrnd9"]
Jul 27 19:41:52.888799 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:41:52.888765    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef961822_0c5d_4af5_9ad6_479b3b414144.slice/crio-107f29c349318996f8ec617a3afb9609275328a02ad02c01ad703f2507d67d2b WatchSource:0}: Error finding container 107f29c349318996f8ec617a3afb9609275328a02ad02c01ad703f2507d67d2b: Status 404 returned error can't find the container with id 107f29c349318996f8ec617a3afb9609275328a02ad02c01ad703f2507d67d2b
Jul 27 19:41:52.890407 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.890382    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 19:41:52.901329 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:52.901299    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-zrnd9" event={"ID":"ef961822-0c5d-4af5-9ad6-479b3b414144","Type":"ContainerStarted","Data":"107f29c349318996f8ec617a3afb9609275328a02ad02c01ad703f2507d67d2b"}
Jul 27 19:41:53.907571 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:53.907533    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-zrnd9" event={"ID":"ef961822-0c5d-4af5-9ad6-479b3b414144","Type":"ContainerStarted","Data":"e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf"}
Jul 27 19:41:53.908013 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:53.907686    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:41:53.924543 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:41:53.924287    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-686ff9dc49-zrnd9" podStartSLOduration=1.336856723 podStartE2EDuration="1.924264983s" totalImagesPullingTime="587.40826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:41:53.923459883 +0000 UTC m=+932.581169412" watchObservedRunningTime="2026-07-27 19:41:53.924264983 +0000 UTC m=+932.581974513"
Jul 27 19:42:04.914261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:04.914231    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:42:09.571950 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.571917    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-zrnd9"]
Jul 27 19:42:09.572428 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.572114    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-686ff9dc49-zrnd9" podUID="ef961822-0c5d-4af5-9ad6-479b3b414144" containerName="manager" containerID="cri-o://e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf" gracePeriod=30
Jul 27 19:42:09.817566 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.817541    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:42:09.951242 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.951214    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2t94\" (UniqueName: \"kubernetes.io/projected/ef961822-0c5d-4af5-9ad6-479b3b414144-kube-api-access-r2t94\") pod \"ef961822-0c5d-4af5-9ad6-479b3b414144\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") "
Jul 27 19:42:09.951417 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.951281    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef961822-0c5d-4af5-9ad6-479b3b414144-webhook-cert\") pod \"ef961822-0c5d-4af5-9ad6-479b3b414144\" (UID: \"ef961822-0c5d-4af5-9ad6-479b3b414144\") "
Jul 27 19:42:09.953292 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.953248    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef961822-0c5d-4af5-9ad6-479b3b414144-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ef961822-0c5d-4af5-9ad6-479b3b414144" (UID: "ef961822-0c5d-4af5-9ad6-479b3b414144"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 19:42:09.953418 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.953291    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef961822-0c5d-4af5-9ad6-479b3b414144-kube-api-access-r2t94" (OuterVolumeSpecName: "kube-api-access-r2t94") pod "ef961822-0c5d-4af5-9ad6-479b3b414144" (UID: "ef961822-0c5d-4af5-9ad6-479b3b414144"). InnerVolumeSpecName "kube-api-access-r2t94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:42:09.987858 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.987821    2544 generic.go:356] "Generic (PLEG): container finished" podID="ef961822-0c5d-4af5-9ad6-479b3b414144" containerID="e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf" exitCode=0
Jul 27 19:42:09.987858 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.987865    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-zrnd9"
Jul 27 19:42:09.988058 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.987907    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-zrnd9" event={"ID":"ef961822-0c5d-4af5-9ad6-479b3b414144","Type":"ContainerDied","Data":"e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf"}
Jul 27 19:42:09.988058 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.987976    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-zrnd9" event={"ID":"ef961822-0c5d-4af5-9ad6-479b3b414144","Type":"ContainerDied","Data":"107f29c349318996f8ec617a3afb9609275328a02ad02c01ad703f2507d67d2b"}
Jul 27 19:42:09.988058 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.988019    2544 scope.go:117] "RemoveContainer" containerID="e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf"
Jul 27 19:42:09.997503 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.997483    2544 scope.go:117] "RemoveContainer" containerID="e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf"
Jul 27 19:42:09.997812 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:42:09.997793    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf\": container with ID starting with e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf not found: ID does not exist" containerID="e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf"
Jul 27 19:42:09.997894 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:09.997820    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf"} err="failed to get container status \"e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf\": rpc error: code = NotFound desc = could not find container \"e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf\": container with ID starting with e4f3434dfd7ca66e5af18d9e9da418d1e2df9aa74c0d1d26f0ff8da4502425bf not found: ID does not exist"
Jul 27 19:42:10.010029 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.010003    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-zrnd9"]
Jul 27 19:42:10.013164 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.013144    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-zrnd9"]
Jul 27 19:42:10.052463 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.052432    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2t94\" (UniqueName: \"kubernetes.io/projected/ef961822-0c5d-4af5-9ad6-479b3b414144-kube-api-access-r2t94\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:42:10.052463 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.052462    2544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef961822-0c5d-4af5-9ad6-479b3b414144-webhook-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:42:10.774993 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.774959    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-686ff9dc49-hvg25"]
Jul 27 19:42:10.775367 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.775346    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef961822-0c5d-4af5-9ad6-479b3b414144" containerName="manager"
Jul 27 19:42:10.775367 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.775356    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef961822-0c5d-4af5-9ad6-479b3b414144" containerName="manager"
Jul 27 19:42:10.775448 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.775437    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef961822-0c5d-4af5-9ad6-479b3b414144" containerName="manager"
Jul 27 19:42:10.779669 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.779651    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:10.783526 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.783502    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 27 19:42:10.783661 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.783503    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7sbpd\""
Jul 27 19:42:10.783893 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.783877    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 27 19:42:10.787958 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.787938    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-hvg25"]
Jul 27 19:42:10.861156 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.861122    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b343eaa-f4c0-4784-976d-4dbbeaec5663-webhook-cert\") pod \"maas-controller-686ff9dc49-hvg25\" (UID: \"3b343eaa-f4c0-4784-976d-4dbbeaec5663\") " pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:10.861331 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.861239    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmmp\" (UniqueName: \"kubernetes.io/projected/3b343eaa-f4c0-4784-976d-4dbbeaec5663-kube-api-access-htmmp\") pod \"maas-controller-686ff9dc49-hvg25\" (UID: \"3b343eaa-f4c0-4784-976d-4dbbeaec5663\") " pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:10.962064 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.962034    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmmp\" (UniqueName: \"kubernetes.io/projected/3b343eaa-f4c0-4784-976d-4dbbeaec5663-kube-api-access-htmmp\") pod \"maas-controller-686ff9dc49-hvg25\" (UID: \"3b343eaa-f4c0-4784-976d-4dbbeaec5663\") " pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:10.962238 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.962087    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b343eaa-f4c0-4784-976d-4dbbeaec5663-webhook-cert\") pod \"maas-controller-686ff9dc49-hvg25\" (UID: \"3b343eaa-f4c0-4784-976d-4dbbeaec5663\") " pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:10.964482 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.964459    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b343eaa-f4c0-4784-976d-4dbbeaec5663-webhook-cert\") pod \"maas-controller-686ff9dc49-hvg25\" (UID: \"3b343eaa-f4c0-4784-976d-4dbbeaec5663\") " pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:10.970506 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:10.970476    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmmp\" (UniqueName: \"kubernetes.io/projected/3b343eaa-f4c0-4784-976d-4dbbeaec5663-kube-api-access-htmmp\") pod \"maas-controller-686ff9dc49-hvg25\" (UID: \"3b343eaa-f4c0-4784-976d-4dbbeaec5663\") " pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:11.091421 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:11.091333    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:11.224785 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:11.224754    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686ff9dc49-hvg25"]
Jul 27 19:42:11.227834 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:42:11.227805    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b343eaa_f4c0_4784_976d_4dbbeaec5663.slice/crio-80a3243e9669e46fd2bee29d3c7c80b5d7d8f2c3f2c296bf15a4fe190c3e7ceb WatchSource:0}: Error finding container 80a3243e9669e46fd2bee29d3c7c80b5d7d8f2c3f2c296bf15a4fe190c3e7ceb: Status 404 returned error can't find the container with id 80a3243e9669e46fd2bee29d3c7c80b5d7d8f2c3f2c296bf15a4fe190c3e7ceb
Jul 27 19:42:11.841764 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:11.841657    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef961822-0c5d-4af5-9ad6-479b3b414144" path="/var/lib/kubelet/pods/ef961822-0c5d-4af5-9ad6-479b3b414144/volumes"
Jul 27 19:42:12.000521 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:12.000479    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-hvg25" event={"ID":"3b343eaa-f4c0-4784-976d-4dbbeaec5663","Type":"ContainerStarted","Data":"8816b4968c43e691e31b9c532d558830a911206f133fe924ae06dc76d12372e7"}
Jul 27 19:42:12.000707 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:12.000555    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686ff9dc49-hvg25" event={"ID":"3b343eaa-f4c0-4784-976d-4dbbeaec5663","Type":"ContainerStarted","Data":"80a3243e9669e46fd2bee29d3c7c80b5d7d8f2c3f2c296bf15a4fe190c3e7ceb"}
Jul 27 19:42:12.000707 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:12.000626    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:42:12.018320 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:12.018269    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-686ff9dc49-hvg25" podStartSLOduration=1.69083505 podStartE2EDuration="2.018253949s" totalImagesPullingTime="327.418899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:42:12.017572816 +0000 UTC m=+950.675282346" watchObservedRunningTime="2026-07-27 19:42:12.018253949 +0000 UTC m=+950.675963476"
Jul 27 19:42:23.008554 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:42:23.008522    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-686ff9dc49-hvg25"
Jul 27 19:46:21.790197 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:46:21.790170    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:46:21.791034 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:46:21.791012    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:51:21.819383 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:51:21.819355    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:51:21.822487 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:51:21.822466    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:55:23.680588 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.680555    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"]
Jul 27 19:55:23.684035 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.684019    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.686980 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.686960    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-aad6c451-dockercfg-2lqfp\""
Jul 27 19:55:23.687093 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.687005    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-aad6c451\""
Jul 27 19:55:23.695069 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.695048    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"]
Jul 27 19:55:23.702416 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.702394    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"]
Jul 27 19:55:23.705681 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.705666    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.713446 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.713395    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"]
Jul 27 19:55:23.801846 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.801812    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12a684e9-817f-4aac-b26a-67f6b7a2402f-plugins-config-volume\") pod \"payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") " pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.802000 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.801862    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.802000 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.801955    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67v9p\" (UniqueName: \"kubernetes.io/projected/12a684e9-817f-4aac-b26a-67f6b7a2402f-kube-api-access-67v9p\") pod \"payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") " pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.802086 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.802012    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvbl\" (UniqueName: \"kubernetes.io/projected/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-kube-api-access-tvvbl\") pod \"payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.902885 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.902853    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12a684e9-817f-4aac-b26a-67f6b7a2402f-plugins-config-volume\") pod \"payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") " pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.903057 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.902902    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.903135 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.903074    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67v9p\" (UniqueName: \"kubernetes.io/projected/12a684e9-817f-4aac-b26a-67f6b7a2402f-kube-api-access-67v9p\") pod \"payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") " pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.903135 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.903117    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvbl\" (UniqueName: \"kubernetes.io/projected/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-kube-api-access-tvvbl\") pod \"payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.903681 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.903659    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12a684e9-817f-4aac-b26a-67f6b7a2402f-plugins-config-volume\") pod \"payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") " pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.903809 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.903660    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.910706 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.910682    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvbl\" (UniqueName: \"kubernetes.io/projected/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-kube-api-access-tvvbl\") pod \"payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:23.910862 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.910847    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67v9p\" (UniqueName: \"kubernetes.io/projected/12a684e9-817f-4aac-b26a-67f6b7a2402f-kube-api-access-67v9p\") pod \"payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") " pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:23.994972 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:23.994915    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:24.016802 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.016777    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:24.146485 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.146229    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"]
Jul 27 19:55:24.148635 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:55:24.148602    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4830a0cd_89ef_46e3_834b_5a5bee5d29c6.slice/crio-d655f5d9b20f6eb4c368dfe0c814db1263a3da6296f527e6d8cc87126b347d50 WatchSource:0}: Error finding container d655f5d9b20f6eb4c368dfe0c814db1263a3da6296f527e6d8cc87126b347d50: Status 404 returned error can't find the container with id d655f5d9b20f6eb4c368dfe0c814db1263a3da6296f527e6d8cc87126b347d50
Jul 27 19:55:24.382827 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.382804    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"]
Jul 27 19:55:24.384830 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:55:24.384797    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a684e9_817f_4aac_b26a_67f6b7a2402f.slice/crio-863ecf48058aebd8f73ea303e856c8d9609a9f654c9d315d4addf86ce118414c WatchSource:0}: Error finding container 863ecf48058aebd8f73ea303e856c8d9609a9f654c9d315d4addf86ce118414c: Status 404 returned error can't find the container with id 863ecf48058aebd8f73ea303e856c8d9609a9f654c9d315d4addf86ce118414c
Jul 27 19:55:24.833022 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.832932    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph" event={"ID":"12a684e9-817f-4aac-b26a-67f6b7a2402f","Type":"ContainerStarted","Data":"7258071f22c41ee936029d080953c2184e7feab65abc97359092d707783bbb2a"}
Jul 27 19:55:24.833424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.833028    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:24.833424 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.833057    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph" event={"ID":"12a684e9-817f-4aac-b26a-67f6b7a2402f","Type":"ContainerStarted","Data":"863ecf48058aebd8f73ea303e856c8d9609a9f654c9d315d4addf86ce118414c"}
Jul 27 19:55:24.834473 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.834450    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd" event={"ID":"4830a0cd-89ef-46e3-834b-5a5bee5d29c6","Type":"ContainerStarted","Data":"4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25"}
Jul 27 19:55:24.834559 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.834495    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd" event={"ID":"4830a0cd-89ef-46e3-834b-5a5bee5d29c6","Type":"ContainerStarted","Data":"d655f5d9b20f6eb4c368dfe0c814db1263a3da6296f527e6d8cc87126b347d50"}
Jul 27 19:55:24.834598 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.834582    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:24.851749 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.851696    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph" podStartSLOduration=1.85168374 podStartE2EDuration="1.85168374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:55:24.851547759 +0000 UTC m=+1743.509257314" watchObservedRunningTime="2026-07-27 19:55:24.85168374 +0000 UTC m=+1743.509393269"
Jul 27 19:55:24.867419 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:24.867352    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd" podStartSLOduration=1.867336643 podStartE2EDuration="1.867336643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:55:24.866634049 +0000 UTC m=+1743.524343579" watchObservedRunningTime="2026-07-27 19:55:24.867336643 +0000 UTC m=+1743.525046175"
Jul 27 19:55:35.842109 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:35.842072    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:35.842521 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:35.842238    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:58.896308 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:58.896267    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"]
Jul 27 19:55:58.896926 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:58.896507    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph" podUID="12a684e9-817f-4aac-b26a-67f6b7a2402f" containerName="payload-processing" containerID="cri-o://7258071f22c41ee936029d080953c2184e7feab65abc97359092d707783bbb2a" gracePeriod=30
Jul 27 19:55:58.903343 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:58.903318    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"]
Jul 27 19:55:58.904561 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:58.903872    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd" podUID="4830a0cd-89ef-46e3-834b-5a5bee5d29c6" containerName="payload-pre-processing" containerID="cri-o://4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25" gracePeriod=30
Jul 27 19:55:59.014088 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.014060    2544 generic.go:356] "Generic (PLEG): container finished" podID="12a684e9-817f-4aac-b26a-67f6b7a2402f" containerID="7258071f22c41ee936029d080953c2184e7feab65abc97359092d707783bbb2a" exitCode=0
Jul 27 19:55:59.014197 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.014132    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph" event={"ID":"12a684e9-817f-4aac-b26a-67f6b7a2402f","Type":"ContainerDied","Data":"7258071f22c41ee936029d080953c2184e7feab65abc97359092d707783bbb2a"}
Jul 27 19:55:59.196127 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.196107    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:55:59.198865 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.198845    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:55:59.283675 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.283649    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvvbl\" (UniqueName: \"kubernetes.io/projected/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-kube-api-access-tvvbl\") pod \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") "
Jul 27 19:55:59.283966 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.283689    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-plugins-config-volume\") pod \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\" (UID: \"4830a0cd-89ef-46e3-834b-5a5bee5d29c6\") "
Jul 27 19:55:59.283966 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.283724    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12a684e9-817f-4aac-b26a-67f6b7a2402f-plugins-config-volume\") pod \"12a684e9-817f-4aac-b26a-67f6b7a2402f\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") "
Jul 27 19:55:59.283966 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.283816    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67v9p\" (UniqueName: \"kubernetes.io/projected/12a684e9-817f-4aac-b26a-67f6b7a2402f-kube-api-access-67v9p\") pod \"12a684e9-817f-4aac-b26a-67f6b7a2402f\" (UID: \"12a684e9-817f-4aac-b26a-67f6b7a2402f\") "
Jul 27 19:55:59.284242 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.284210    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12a684e9-817f-4aac-b26a-67f6b7a2402f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12a684e9-817f-4aac-b26a-67f6b7a2402f" (UID: "12a684e9-817f-4aac-b26a-67f6b7a2402f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:55:59.284351 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.284258    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4830a0cd-89ef-46e3-834b-5a5bee5d29c6" (UID: "4830a0cd-89ef-46e3-834b-5a5bee5d29c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:55:59.285807 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.285786    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12a684e9-817f-4aac-b26a-67f6b7a2402f-kube-api-access-67v9p" (OuterVolumeSpecName: "kube-api-access-67v9p") pod "12a684e9-817f-4aac-b26a-67f6b7a2402f" (UID: "12a684e9-817f-4aac-b26a-67f6b7a2402f"). InnerVolumeSpecName "kube-api-access-67v9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:55:59.285807 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.285799    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-kube-api-access-tvvbl" (OuterVolumeSpecName: "kube-api-access-tvvbl") pod "4830a0cd-89ef-46e3-834b-5a5bee5d29c6" (UID: "4830a0cd-89ef-46e3-834b-5a5bee5d29c6"). InnerVolumeSpecName "kube-api-access-tvvbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:55:59.384470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.384446    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvvbl\" (UniqueName: \"kubernetes.io/projected/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-kube-api-access-tvvbl\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:55:59.384470 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.384467    2544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4830a0cd-89ef-46e3-834b-5a5bee5d29c6-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:55:59.384595 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.384476    2544 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12a684e9-817f-4aac-b26a-67f6b7a2402f-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:55:59.384595 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:55:59.384484    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67v9p\" (UniqueName: \"kubernetes.io/projected/12a684e9-817f-4aac-b26a-67f6b7a2402f-kube-api-access-67v9p\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:00.019684 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.019654    2544 generic.go:356] "Generic (PLEG): container finished" podID="4830a0cd-89ef-46e3-834b-5a5bee5d29c6" containerID="4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25" exitCode=0
Jul 27 19:56:00.020059 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.019706    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"
Jul 27 19:56:00.020059 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.019765    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd" event={"ID":"4830a0cd-89ef-46e3-834b-5a5bee5d29c6","Type":"ContainerDied","Data":"4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25"}
Jul 27 19:56:00.020059 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.019831    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd" event={"ID":"4830a0cd-89ef-46e3-834b-5a5bee5d29c6","Type":"ContainerDied","Data":"d655f5d9b20f6eb4c368dfe0c814db1263a3da6296f527e6d8cc87126b347d50"}
Jul 27 19:56:00.020059 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.019863    2544 scope.go:117] "RemoveContainer" containerID="4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25"
Jul 27 19:56:00.021282 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.021262    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph" event={"ID":"12a684e9-817f-4aac-b26a-67f6b7a2402f","Type":"ContainerDied","Data":"863ecf48058aebd8f73ea303e856c8d9609a9f654c9d315d4addf86ce118414c"}
Jul 27 19:56:00.021421 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.021297    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"
Jul 27 19:56:00.028951 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.028930    2544 scope.go:117] "RemoveContainer" containerID="4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25"
Jul 27 19:56:00.029263 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:56:00.029233    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25\": container with ID starting with 4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25 not found: ID does not exist" containerID="4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25"
Jul 27 19:56:00.029361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.029270    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25"} err="failed to get container status \"4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25\": rpc error: code = NotFound desc = could not find container \"4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25\": container with ID starting with 4b448095cbbc78951e08593918ef267d93a262672e13bdb09ce2adefc7412f25 not found: ID does not exist"
Jul 27 19:56:00.029361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.029288    2544 scope.go:117] "RemoveContainer" containerID="7258071f22c41ee936029d080953c2184e7feab65abc97359092d707783bbb2a"
Jul 27 19:56:00.038766 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.038746    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"]
Jul 27 19:56:00.045642 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.045622    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aad6c451-ddcf4774-t2zph"]
Jul 27 19:56:00.054602 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.054585    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"]
Jul 27 19:56:00.057816 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.057797    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-aad6c451-57f65ffdd8-kjbvd"]
Jul 27 19:56:00.181419 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.181387    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"]
Jul 27 19:56:00.181859 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.181842    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4830a0cd-89ef-46e3-834b-5a5bee5d29c6" containerName="payload-pre-processing"
Jul 27 19:56:00.181951 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.181862    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="4830a0cd-89ef-46e3-834b-5a5bee5d29c6" containerName="payload-pre-processing"
Jul 27 19:56:00.181951 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.181880    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12a684e9-817f-4aac-b26a-67f6b7a2402f" containerName="payload-processing"
Jul 27 19:56:00.181951 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.181889    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="12a684e9-817f-4aac-b26a-67f6b7a2402f" containerName="payload-processing"
Jul 27 19:56:00.182144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.181983    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="12a684e9-817f-4aac-b26a-67f6b7a2402f" containerName="payload-processing"
Jul 27 19:56:00.182144 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.182001    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="4830a0cd-89ef-46e3-834b-5a5bee5d29c6" containerName="payload-pre-processing"
Jul 27 19:56:00.186804 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.186785    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189149 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189131    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f82daff9-gw-openshift-default-dockercfg-2z8xn\""
Jul 27 19:56:00.189267 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189244    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwrx6\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-kube-api-access-pwrx6\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189318 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189287    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-credential-socket\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189357 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189334    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istiod-ca-cert\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189430 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189416    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-certs\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189483 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189467    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-socket\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189517 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189499    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-envoy\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189555 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189529    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-token\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189589 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189555    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-data\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.189622 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.189616    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-podinfo\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.197358 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.197339    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"]
Jul 27 19:56:00.290679 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.290651    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-certs\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.290835 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.290697    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-socket\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.290928 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.290899    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-envoy\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291010 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.290948    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-token\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291010 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.290978    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-data\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291126 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291023    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-podinfo\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291126 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291062    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwrx6\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-kube-api-access-pwrx6\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291126 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291079    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-socket\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291126 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291103    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-credential-socket\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291333 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291153    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istiod-ca-cert\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291333 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291180    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-certs\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291609 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291573    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-data\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.291764 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.291740    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-credential-socket\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.292058 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.292036    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istiod-ca-cert\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.293570 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.293540    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-podinfo\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.293706 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.293690    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-envoy\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.298942 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.298920    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-token\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.299050 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.299034    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwrx6\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-kube-api-access-pwrx6\") pod \"e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") " pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.497364 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.497307    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:00.634998 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.634973    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"]
Jul 27 19:56:00.637262 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:56:00.637238    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f754ed_fcc1_436e_8e2c_fe431cdced1e.slice/crio-420f80b19366ec7eedacdb1dacb4e029844b13038f4a7728c0a794c804db36e0 WatchSource:0}: Error finding container 420f80b19366ec7eedacdb1dacb4e029844b13038f4a7728c0a794c804db36e0: Status 404 returned error can't find the container with id 420f80b19366ec7eedacdb1dacb4e029844b13038f4a7728c0a794c804db36e0
Jul 27 19:56:00.639268 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.639236    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:56:00.639357 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.639298    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:56:00.639357 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:00.639325    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:56:01.028998 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.028964    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" event={"ID":"59f754ed-fcc1-436e-8e2c-fe431cdced1e","Type":"ContainerStarted","Data":"f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"}
Jul 27 19:56:01.029372 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.029021    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" event={"ID":"59f754ed-fcc1-436e-8e2c-fe431cdced1e","Type":"ContainerStarted","Data":"420f80b19366ec7eedacdb1dacb4e029844b13038f4a7728c0a794c804db36e0"}
Jul 27 19:56:01.047660 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.047618    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podStartSLOduration=1.047604149 podStartE2EDuration="1.047604149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:56:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:56:01.046754845 +0000 UTC m=+1779.704464367" watchObservedRunningTime="2026-07-27 19:56:01.047604149 +0000 UTC m=+1779.705313745"
Jul 27 19:56:01.498350 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.498323    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:01.502967 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.502936    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:01.840588 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.840518    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12a684e9-817f-4aac-b26a-67f6b7a2402f" path="/var/lib/kubelet/pods/12a684e9-817f-4aac-b26a-67f6b7a2402f/volumes"
Jul 27 19:56:01.840902 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:01.840889    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4830a0cd-89ef-46e3-834b-5a5bee5d29c6" path="/var/lib/kubelet/pods/4830a0cd-89ef-46e3-834b-5a5bee5d29c6/volumes"
Jul 27 19:56:02.034332 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:02.034302    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:02.035079 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:02.035062    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:17.463085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:17.462806    2544 patch_prober.go:28] interesting pod/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:42982->10.132.0.54:15021: read: connection reset by peer" start-of-body=
Jul 27 19:56:17.463085 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:17.462868    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:42982->10.132.0.54:15021: read: connection reset by peer"
Jul 27 19:56:18.120439 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:18.120387    2544 generic.go:356] "Generic (PLEG): container finished" podID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerID="f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef" exitCode=0
Jul 27 19:56:18.120613 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:18.120473    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" event={"ID":"59f754ed-fcc1-436e-8e2c-fe431cdced1e","Type":"ContainerDied","Data":"f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"}
Jul 27 19:56:18.120925 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:18.120907    2544 scope.go:117] "RemoveContainer" containerID="f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"
Jul 27 19:56:18.122047 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:18.122015    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:56:18.122118 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:18.122103    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:56:18.122156 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:18.122146    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:56:19.127279 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:19.127242    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" event={"ID":"59f754ed-fcc1-436e-8e2c-fe431cdced1e","Type":"ContainerStarted","Data":"21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d"}
Jul 27 19:56:19.498092 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:19.498067    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:19.498287 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:19.498266    2544 patch_prober.go:28] interesting pod/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 27 19:56:19.498363 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:19.498333    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 27 19:56:20.498051 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:20.498017    2544 patch_prober.go:28] interesting pod/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 27 19:56:20.498508 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:20.498086    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 27 19:56:21.498099 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:21.498068    2544 patch_prober.go:28] interesting pod/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 27 19:56:21.498464 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:21.498122    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 27 19:56:21.850959 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:21.850876    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:56:21.854810 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:21.854789    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 19:56:22.497942 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:22.497907    2544 patch_prober.go:28] interesting pod/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 27 19:56:22.498103 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:22.497959    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 27 19:56:23.501162 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:23.501133    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:23.501526 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:23.501392    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:23.502140 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:23.502121    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:38.202090 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:38.202055    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"]
Jul 27 19:56:38.202854 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:38.202819    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" containerID="cri-o://21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d" gracePeriod=30
Jul 27 19:56:38.503719 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:38.503645    2544 patch_prober.go:28] interesting pod/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 19:56:38.503719 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:38.503706    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 19:56:43.458415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.458387    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:43.573639 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573565    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-token\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573776 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573654    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-credential-socket\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573776 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573677    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-socket\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573776 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573702    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwrx6\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-kube-api-access-pwrx6\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573776 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573720    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-certs\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573795    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-envoy\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573831    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istiod-ca-cert\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573853    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-data\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.573971 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573911    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-podinfo\") pod \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\" (UID: \"59f754ed-fcc1-436e-8e2c-fe431cdced1e\") "
Jul 27 19:56:43.574164 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.573974    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:56:43.574164 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.574003    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:56:43.574277 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.574238    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.574277 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.574259    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.574375 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.574278    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:56:43.574375 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.574346    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:56:43.575176 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.575145    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:56:43.575878 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.575851    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 19:56:43.575996 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.575969    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-kube-api-access-pwrx6" (OuterVolumeSpecName: "kube-api-access-pwrx6") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "kube-api-access-pwrx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:56:43.576107 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.576085    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:56:43.576160 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.576118    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "59f754ed-fcc1-436e-8e2c-fe431cdced1e" (UID: "59f754ed-fcc1-436e-8e2c-fe431cdced1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:56:43.675207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675185    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.675207 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675206    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwrx6\" (UniqueName: \"kubernetes.io/projected/59f754ed-fcc1-436e-8e2c-fe431cdced1e-kube-api-access-pwrx6\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.675341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675216    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.675341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675225    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.675341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675231    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.675341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675238    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:43.675341 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:43.675245    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f754ed-fcc1-436e-8e2c-fe431cdced1e-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:56:44.267786 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.267721    2544 generic.go:356] "Generic (PLEG): container finished" podID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerID="21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d" exitCode=0
Jul 27 19:56:44.267786 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.267762    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" event={"ID":"59f754ed-fcc1-436e-8e2c-fe431cdced1e","Type":"ContainerDied","Data":"21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d"}
Jul 27 19:56:44.267955 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.267791    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"
Jul 27 19:56:44.267955 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.267823    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f" event={"ID":"59f754ed-fcc1-436e-8e2c-fe431cdced1e","Type":"ContainerDied","Data":"420f80b19366ec7eedacdb1dacb4e029844b13038f4a7728c0a794c804db36e0"}
Jul 27 19:56:44.267955 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.267847    2544 scope.go:117] "RemoveContainer" containerID="21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d"
Jul 27 19:56:44.277350 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.277303    2544 scope.go:117] "RemoveContainer" containerID="f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"
Jul 27 19:56:44.285821 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.285799    2544 scope.go:117] "RemoveContainer" containerID="21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d"
Jul 27 19:56:44.286105 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:56:44.286082    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d\": container with ID starting with 21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d not found: ID does not exist" containerID="21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d"
Jul 27 19:56:44.286219 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.286104    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"]
Jul 27 19:56:44.286219 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.286109    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d"} err="failed to get container status \"21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d\": rpc error: code = NotFound desc = could not find container \"21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d\": container with ID starting with 21199066f64e042c9c712a323bc8d2f3e684b490f7729aaf1280d107415b727d not found: ID does not exist"
Jul 27 19:56:44.286219 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.286126    2544 scope.go:117] "RemoveContainer" containerID="f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"
Jul 27 19:56:44.286397 ip-10-0-128-237 kubenswrapper[2544]: E0727 19:56:44.286382    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef\": container with ID starting with f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef not found: ID does not exist" containerID="f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"
Jul 27 19:56:44.286434 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.286399    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef"} err="failed to get container status \"f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef\": rpc error: code = NotFound desc = could not find container \"f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef\": container with ID starting with f0c344afd8d80475daf15504b2c04a6dda56aa9c615efb26a7b073b80ab86eef not found: ID does not exist"
Jul 27 19:56:44.292180 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:44.292157    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f82daff9-gw-openshift-default-5cb7fb7484-89j5f"]
Jul 27 19:56:45.840710 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:56:45.840680    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" path="/var/lib/kubelet/pods/59f754ed-fcc1-436e-8e2c-fe431cdced1e/volumes"
Jul 27 19:57:19.553851 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.553807    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"]
Jul 27 19:57:19.554291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.554245    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy"
Jul 27 19:57:19.554291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.554261    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy"
Jul 27 19:57:19.554361 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.554342    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy"
Jul 27 19:57:19.557091 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.557072    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.559830 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.559810    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ee5db6ec-openshift-default-dockercfg-s96mh\""
Jul 27 19:57:19.569617 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.569595    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"]
Jul 27 19:57:19.664094 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664063    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-credential-socket\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664266 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664112    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-envoy\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664266 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664175    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-socket\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664266 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664223    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99b86817-428e-4168-bf91-de69bdd933ce-istio-podinfo\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664266 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664251    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-istio-token\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664287    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99b86817-428e-4168-bf91-de69bdd933ce-istiod-ca-cert\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664331    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-data\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664382    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-certs\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.664505 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.664418    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkwbg\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-kube-api-access-tkwbg\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.765797 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765763    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-socket\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.765960 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765811    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99b86817-428e-4168-bf91-de69bdd933ce-istio-podinfo\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.765960 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765842    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-istio-token\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.765960 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765877    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99b86817-428e-4168-bf91-de69bdd933ce-istiod-ca-cert\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.765960 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765904    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-data\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.765960 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765943    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-certs\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766216 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.765972    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkwbg\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-kube-api-access-tkwbg\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766216 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766026    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-credential-socket\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766216 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766065    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-envoy\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766216 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766201    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-socket\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766755 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766441    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-certs\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766755 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766690    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-credential-socket\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766755 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766703    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-data\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.766978 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.766851    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99b86817-428e-4168-bf91-de69bdd933ce-istiod-ca-cert\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.768211 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.768182    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-envoy\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.768518 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.768499    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99b86817-428e-4168-bf91-de69bdd933ce-istio-podinfo\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.774291 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.774271    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-istio-token\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.774415 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.774392    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkwbg\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-kube-api-access-tkwbg\") pod \"e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") " pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:19.867684 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:19.867655    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:20.004523 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.004493    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"]
Jul 27 19:57:20.005765 ip-10-0-128-237 kubenswrapper[2544]: W0727 19:57:20.005717    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99b86817_428e_4168_bf91_de69bdd933ce.slice/crio-9d2f4093c63b1e2b4639449541af8af6789a9f873411b83096e517c8752de4db WatchSource:0}: Error finding container 9d2f4093c63b1e2b4639449541af8af6789a9f873411b83096e517c8752de4db: Status 404 returned error can't find the container with id 9d2f4093c63b1e2b4639449541af8af6789a9f873411b83096e517c8752de4db
Jul 27 19:57:20.007778 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.007748    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:57:20.007900 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.007810    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:57:20.007900 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.007838    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:57:20.449052 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.449019    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" event={"ID":"99b86817-428e-4168-bf91-de69bdd933ce","Type":"ContainerStarted","Data":"75258d9a72810008911095639516a1754831a33ff5531776d8685edfffc71fb5"}
Jul 27 19:57:20.449205 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.449070    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" event={"ID":"99b86817-428e-4168-bf91-de69bdd933ce","Type":"ContainerStarted","Data":"9d2f4093c63b1e2b4639449541af8af6789a9f873411b83096e517c8752de4db"}
Jul 27 19:57:20.468789 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.468720    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" podStartSLOduration=1.468704522 podStartE2EDuration="1.468704522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 19:57:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 19:57:20.467034577 +0000 UTC m=+1859.124744108" watchObservedRunningTime="2026-07-27 19:57:20.468704522 +0000 UTC m=+1859.126414051"
Jul 27 19:57:20.867793 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:20.867771    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:21.873862 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:21.873834    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:22.459068 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:22.459039    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:22.459814 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:22.459794    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:35.535157 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:35.535129    2544 generic.go:356] "Generic (PLEG): container finished" podID="99b86817-428e-4168-bf91-de69bdd933ce" containerID="75258d9a72810008911095639516a1754831a33ff5531776d8685edfffc71fb5" exitCode=0
Jul 27 19:57:35.535497 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:35.535212    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" event={"ID":"99b86817-428e-4168-bf91-de69bdd933ce","Type":"ContainerDied","Data":"75258d9a72810008911095639516a1754831a33ff5531776d8685edfffc71fb5"}
Jul 27 19:57:35.535585 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:35.535570    2544 scope.go:117] "RemoveContainer" containerID="75258d9a72810008911095639516a1754831a33ff5531776d8685edfffc71fb5"
Jul 27 19:57:35.536694 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:35.536659    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:57:35.536809 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:35.536770    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:57:35.536859 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:35.536813    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 19:57:36.544889 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:36.544855    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" event={"ID":"99b86817-428e-4168-bf91-de69bdd933ce","Type":"ContainerStarted","Data":"ee404083282eadaf79265e1776f6b2b1c4cc789fa5e58d75489d97350c9c5b43"}
Jul 27 19:57:36.868323 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:36.868279    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:36.869588 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:36.869562    2544 patch_prober.go:28] interesting pod/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 19:57:36.869843 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:36.869802    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 19:57:37.459805 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:37.459780    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:37.868877 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:37.868839    2544 patch_prober.go:28] interesting pod/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 19:57:37.869330 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:37.868896    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 19:57:38.869100 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:38.869045    2544 patch_prober.go:28] interesting pod/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 19:57:38.869550 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:38.869120    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 19:57:39.868041 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:39.868007    2544 patch_prober.go:28] interesting pod/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 27 19:57:39.868668 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:39.868294    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 27 19:57:40.872261 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:40.872226    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:40.873564 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:40.873537    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:57:57.551939 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:57.551903    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"]
Jul 27 19:57:57.552425 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:57:57.552152    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy" containerID="cri-o://ee404083282eadaf79265e1776f6b2b1c4cc789fa5e58d75489d97350c9c5b43" gracePeriod=30
Jul 27 19:58:02.694201 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.694162    2544 generic.go:356] "Generic (PLEG): container finished" podID="99b86817-428e-4168-bf91-de69bdd933ce" containerID="ee404083282eadaf79265e1776f6b2b1c4cc789fa5e58d75489d97350c9c5b43" exitCode=0
Jul 27 19:58:02.694552 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.694233    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" event={"ID":"99b86817-428e-4168-bf91-de69bdd933ce","Type":"ContainerDied","Data":"ee404083282eadaf79265e1776f6b2b1c4cc789fa5e58d75489d97350c9c5b43"}
Jul 27 19:58:02.694552 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.694297    2544 scope.go:117] "RemoveContainer" containerID="75258d9a72810008911095639516a1754831a33ff5531776d8685edfffc71fb5"
Jul 27 19:58:02.801617 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.801596    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:58:02.934303 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934279    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-istio-token\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934432 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934315    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99b86817-428e-4168-bf91-de69bdd933ce-istiod-ca-cert\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934432 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934354    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-socket\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934540 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934469    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkwbg\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-kube-api-access-tkwbg\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934540 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934520    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99b86817-428e-4168-bf91-de69bdd933ce-istio-podinfo\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934651 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934550    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-certs\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934651 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934590    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:58:02.934651 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934591    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-credential-socket\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934651 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934645    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-data\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934706    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-envoy\") pod \"99b86817-428e-4168-bf91-de69bdd933ce\" (UID: \"99b86817-428e-4168-bf91-de69bdd933ce\") "
Jul 27 19:58:02.934937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934709    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99b86817-428e-4168-bf91-de69bdd933ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 19:58:02.934937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934831    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:58:02.934937 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.934850    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:58:02.935146 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.935053    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99b86817-428e-4168-bf91-de69bdd933ce-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:02.935146 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.935072    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:02.935146 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.935086    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:02.935146 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.935098    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:02.935619 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.935600    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:58:02.936943 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.936917    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:58:02.936943 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.936917    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99b86817-428e-4168-bf91-de69bdd933ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 19:58:02.937072 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.936977    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 19:58:02.937072 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:02.936989    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-kube-api-access-tkwbg" (OuterVolumeSpecName: "kube-api-access-tkwbg") pod "99b86817-428e-4168-bf91-de69bdd933ce" (UID: "99b86817-428e-4168-bf91-de69bdd933ce"). InnerVolumeSpecName "kube-api-access-tkwbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 19:58:03.035771 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.035750    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:03.035771 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.035769    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99b86817-428e-4168-bf91-de69bdd933ce-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:03.035895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.035778    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:03.035895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.035786    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkwbg\" (UniqueName: \"kubernetes.io/projected/99b86817-428e-4168-bf91-de69bdd933ce-kube-api-access-tkwbg\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:03.035895 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.035795    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99b86817-428e-4168-bf91-de69bdd933ce-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 19:58:03.700922 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.700902    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"
Jul 27 19:58:03.701297 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.700903    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw" event={"ID":"99b86817-428e-4168-bf91-de69bdd933ce","Type":"ContainerDied","Data":"9d2f4093c63b1e2b4639449541af8af6789a9f873411b83096e517c8752de4db"}
Jul 27 19:58:03.701297 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.701021    2544 scope.go:117] "RemoveContainer" containerID="ee404083282eadaf79265e1776f6b2b1c4cc789fa5e58d75489d97350c9c5b43"
Jul 27 19:58:03.722305 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.722284    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"]
Jul 27 19:58:03.726159 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.726135    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ee5db6ec-openshift-default-79ddf6f64d-zkpkw"]
Jul 27 19:58:03.841210 ip-10-0-128-237 kubenswrapper[2544]: I0727 19:58:03.841188    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99b86817-428e-4168-bf91-de69bdd933ce" path="/var/lib/kubelet/pods/99b86817-428e-4168-bf91-de69bdd933ce/volumes"
Jul 27 20:01:21.881852 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:01:21.881825    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:01:21.896853 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:01:21.896835    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:03:30.837649 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.837621    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"]
Jul 27 20:03:30.838272 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838157    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy"
Jul 27 20:03:30.838272 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838176    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy"
Jul 27 20:03:30.838272 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838189    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy"
Jul 27 20:03:30.838272 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838198    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy"
Jul 27 20:03:30.838494 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838295    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy"
Jul 27 20:03:30.838494 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838303    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy"
Jul 27 20:03:30.838494 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.838310    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f754ed-fcc1-436e-8e2c-fe431cdced1e" containerName="istio-proxy"
Jul 27 20:03:30.841336 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.841313    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.843783 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.843763    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-08e715-openshift-default-dockercfg-55l97\""
Jul 27 20:03:30.853122 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.853100    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"]
Jul 27 20:03:30.864073 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864050    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-socket\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864181 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864092    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-podinfo\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864181 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864115    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-envoy\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864181 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864148    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-credential-socket\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864369 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864197    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-certs\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864369 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864265    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istiod-ca-cert\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864369 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864330    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-data\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864507 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864373    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvwjz\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-kube-api-access-lvwjz\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.864507 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.864405    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-token\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965681 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965650    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-certs\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965832 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965720    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istiod-ca-cert\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965832 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965775    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-data\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965832 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965805    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvwjz\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-kube-api-access-lvwjz\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965986 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965835    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-token\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965986 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965908    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-socket\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965986 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965943    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-podinfo\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.965986 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.965976    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-envoy\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.966190 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.966019    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-credential-socket\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.966190 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.966027    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-certs\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.966330 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.966283    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-socket\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.966457 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.966336    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-credential-socket\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.966457 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.966372    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-data\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.966457 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.966378    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istiod-ca-cert\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.968456 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.968427    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-podinfo\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.968576 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.968559    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-envoy\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.973080 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.973043    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-token\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:30.973708 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:30.973685    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvwjz\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-kube-api-access-lvwjz\") pod \"e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") " pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:31.151919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:31.151891    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:31.303276 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:31.303249    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"]
Jul 27 20:03:31.305349 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:03:31.305319    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f85add_f2f1_4d7c_b46f_4aae3cf9230c.slice/crio-3b556044fa9c5d5a872f6ad34a8f2679f044a717cba6d04ba3ee44b718d99f20 WatchSource:0}: Error finding container 3b556044fa9c5d5a872f6ad34a8f2679f044a717cba6d04ba3ee44b718d99f20: Status 404 returned error can't find the container with id 3b556044fa9c5d5a872f6ad34a8f2679f044a717cba6d04ba3ee44b718d99f20
Jul 27 20:03:31.307252 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:31.307223    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:03:31.307339 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:31.307283    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:03:31.307339 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:31.307310    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:03:31.346493 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:31.346466    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" event={"ID":"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c","Type":"ContainerStarted","Data":"3b556044fa9c5d5a872f6ad34a8f2679f044a717cba6d04ba3ee44b718d99f20"}
Jul 27 20:03:32.352878 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:32.352842    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" event={"ID":"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c","Type":"ContainerStarted","Data":"3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"}
Jul 27 20:03:32.375708 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:32.375661    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" podStartSLOduration=2.37564701 podStartE2EDuration="2.37564701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:03:32.374278995 +0000 UTC m=+2231.031988537" watchObservedRunningTime="2026-07-27 20:03:32.37564701 +0000 UTC m=+2231.033356537"
Jul 27 20:03:33.152168 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:33.152137    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:33.157122 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:33.157101    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:33.358034 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:33.358008    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:33.358767 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:33.358725    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:48.358990 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.358957    2544 patch_prober.go:28] interesting pod/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 27 20:03:48.359361 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.359012    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 27 20:03:48.434234 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.434206    2544 generic.go:356] "Generic (PLEG): container finished" podID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerID="3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3" exitCode=0
Jul 27 20:03:48.434366 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.434276    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" event={"ID":"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c","Type":"ContainerDied","Data":"3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"}
Jul 27 20:03:48.434625 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.434610    2544 scope.go:117] "RemoveContainer" containerID="3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"
Jul 27 20:03:48.435635 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.435605    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:03:48.435716 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.435698    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:03:48.435820 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:48.435791    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:03:49.442270 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:49.442239    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" event={"ID":"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c","Type":"ContainerStarted","Data":"9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283"}
Jul 27 20:03:50.152966 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:50.152932    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:50.153142 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:50.153125    2544 patch_prober.go:28] interesting pod/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 27 20:03:50.153192 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:50.153168    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 27 20:03:51.153047 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:51.153018    2544 patch_prober.go:28] interesting pod/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 27 20:03:51.153421 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:51.153070    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 27 20:03:52.153127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:52.153094    2544 patch_prober.go:28] interesting pod/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 27 20:03:52.153501 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:52.153158    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 27 20:03:53.156058 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:53.156024    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:53.156427 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:53.156272    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:03:53.156878 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:03:53.156861    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:05:39.238747 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:39.238656    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"]
Jul 27 20:05:39.239280 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:39.238967    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy" containerID="cri-o://9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283" gracePeriod=30
Jul 27 20:05:44.508965 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.508909    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:05:44.653723 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653695    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-envoy\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.653912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653772    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-socket\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.653912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653789    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-podinfo\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.653912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653806    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvwjz\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-kube-api-access-lvwjz\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.653912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653834    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-data\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.653912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653851    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-credential-socket\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.653912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653894    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-certs\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.654225 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653927    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-token\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.654225 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.653983    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istiod-ca-cert\") pod \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\" (UID: \"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c\") "
Jul 27 20:05:44.654225 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.654139    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:44.654366 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.654294    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:44.654366 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.654330    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.654475 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.654458    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:05:44.654680 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.654651    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:44.654828 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.654748    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-data" (OuterVolumeSpecName: "istio-data") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:44.656040 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.656016    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:05:44.656151 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.656134    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-kube-api-access-lvwjz" (OuterVolumeSpecName: "kube-api-access-lvwjz") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "kube-api-access-lvwjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:44.656343 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.656324    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:05:44.656420 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.656336    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-token" (OuterVolumeSpecName: "istio-token") pod "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" (UID: "c6f85add-f2f1-4d7c-b46f-4aae3cf9230c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:05:44.755721 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755698    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755721 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755720    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755886 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755760    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755886 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755775    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755886 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755788    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755886 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755800    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755886 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755812    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:44.755886 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:44.755823    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvwjz\" (UniqueName: \"kubernetes.io/projected/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c-kube-api-access-lvwjz\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:05:45.018046 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.018012    2544 generic.go:356] "Generic (PLEG): container finished" podID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerID="9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283" exitCode=0
Jul 27 20:05:45.018235 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.018057    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" event={"ID":"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c","Type":"ContainerDied","Data":"9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283"}
Jul 27 20:05:45.018235 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.018132    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9" event={"ID":"c6f85add-f2f1-4d7c-b46f-4aae3cf9230c","Type":"ContainerDied","Data":"3b556044fa9c5d5a872f6ad34a8f2679f044a717cba6d04ba3ee44b718d99f20"}
Jul 27 20:05:45.018235 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.018068    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"
Jul 27 20:05:45.018406 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.018162    2544 scope.go:117] "RemoveContainer" containerID="9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283"
Jul 27 20:05:45.028295 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.028279    2544 scope.go:117] "RemoveContainer" containerID="3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"
Jul 27 20:05:45.038500 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.038468    2544 scope.go:117] "RemoveContainer" containerID="9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283"
Jul 27 20:05:45.038925 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:05:45.038903    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283\": container with ID starting with 9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283 not found: ID does not exist" containerID="9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283"
Jul 27 20:05:45.039022 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.038932    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283"} err="failed to get container status \"9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283\": rpc error: code = NotFound desc = could not find container \"9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283\": container with ID starting with 9bef005501f7a458236bb803d78d0c9d40497d521fb33fa3eef6a719e8117283 not found: ID does not exist"
Jul 27 20:05:45.039022 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.038949    2544 scope.go:117] "RemoveContainer" containerID="3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"
Jul 27 20:05:45.039204 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:05:45.039175    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3\": container with ID starting with 3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3 not found: ID does not exist" containerID="3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"
Jul 27 20:05:45.039204 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.039198    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3"} err="failed to get container status \"3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3\": rpc error: code = NotFound desc = could not find container \"3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3\": container with ID starting with 3d2984d28faddb2094019261c4e7f71fb379e4c4fd76e44fbb99706a68eac8b3 not found: ID does not exist"
Jul 27 20:05:45.040718 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.040700    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"]
Jul 27 20:05:45.044030 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.044009    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-08e715-openshift-default-b59cd9464-9xqn9"]
Jul 27 20:05:45.842189 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:45.842158    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" path="/var/lib/kubelet/pods/c6f85add-f2f1-4d7c-b46f-4aae3cf9230c/volumes"
Jul 27 20:05:48.680893 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.680860    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"]
Jul 27 20:05:48.681321 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681307    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy"
Jul 27 20:05:48.681364 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681322    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy"
Jul 27 20:05:48.681364 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681336    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy"
Jul 27 20:05:48.681364 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681342    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="99b86817-428e-4168-bf91-de69bdd933ce" containerName="istio-proxy"
Jul 27 20:05:48.681364 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681349    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy"
Jul 27 20:05:48.681364 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681358    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy"
Jul 27 20:05:48.681521 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681441    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy"
Jul 27 20:05:48.681521 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.681450    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6f85add-f2f1-4d7c-b46f-4aae3cf9230c" containerName="istio-proxy"
Jul 27 20:05:48.685007 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.684982    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.687423 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.687400    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d978f1-openshift-default-dockercfg-rvdbb\""
Jul 27 20:05:48.700914 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.700887    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"]
Jul 27 20:05:48.796422 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796393    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-credential-socket\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796422 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796426    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkdc\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-kube-api-access-bqkdc\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796675 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796476    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-socket\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796675 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796560    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-data\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796675 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796630    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istiod-ca-cert\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796675 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796672    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-podinfo\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796742    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-certs\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796778    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-token\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.796912 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.796819    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-envoy\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897437 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897407    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-socket\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897443    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-data\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897462    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istiod-ca-cert\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897491    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-podinfo\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897513    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-certs\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897531    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-token\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897555    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-envoy\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897583    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-credential-socket\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897993 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897664    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkdc\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-kube-api-access-bqkdc\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897993 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897852    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-socket\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.897993 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.897934    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-certs\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.898153 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.898077    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-credential-socket\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.898208 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.898158    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-data\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.898247 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.898201    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istiod-ca-cert\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.900249 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.900222    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-envoy\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.900435 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.900419    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-podinfo\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.907331 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.907312    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-token\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.907605 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.907583    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkdc\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-kube-api-access-bqkdc\") pod \"e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") " pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:48.996935 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:48.996854    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:49.134466 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:49.134442    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"]
Jul 27 20:05:49.137098 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:05:49.137072    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba79523_3c6f_4c4d_888a_3c8be5c24862.slice/crio-10aaecbf26bd4051024b48e4a5e727cc728b3963558536ef8a6f521fe75ad39f WatchSource:0}: Error finding container 10aaecbf26bd4051024b48e4a5e727cc728b3963558536ef8a6f521fe75ad39f: Status 404 returned error can't find the container with id 10aaecbf26bd4051024b48e4a5e727cc728b3963558536ef8a6f521fe75ad39f
Jul 27 20:05:49.139465 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:49.139433    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:05:49.139546 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:49.139511    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:05:49.139582 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:49.139548    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:05:50.046665 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:50.046634    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" event={"ID":"0ba79523-3c6f-4c4d-888a-3c8be5c24862","Type":"ContainerStarted","Data":"b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"}
Jul 27 20:05:50.047058 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:50.046687    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" event={"ID":"0ba79523-3c6f-4c4d-888a-3c8be5c24862","Type":"ContainerStarted","Data":"10aaecbf26bd4051024b48e4a5e727cc728b3963558536ef8a6f521fe75ad39f"}
Jul 27 20:05:50.067892 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:50.067842    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" podStartSLOduration=2.067826765 podStartE2EDuration="2.067826765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:05:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:05:50.066912201 +0000 UTC m=+2368.724621729" watchObservedRunningTime="2026-07-27 20:05:50.067826765 +0000 UTC m=+2368.725536358"
Jul 27 20:05:50.997445 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:50.997413    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:51.002251 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:51.002224    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:51.052083 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:51.052059    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:05:51.052815 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:05:51.052795    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:07.094885 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:07.094844    2544 patch_prober.go:28] interesting pod/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 27 20:06:07.095334 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:07.094926    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 27 20:06:08.151477 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:08.151411    2544 generic.go:356] "Generic (PLEG): container finished" podID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerID="b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81" exitCode=0
Jul 27 20:06:08.151898 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:08.151484    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" event={"ID":"0ba79523-3c6f-4c4d-888a-3c8be5c24862","Type":"ContainerDied","Data":"b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"}
Jul 27 20:06:08.151898 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:08.151892    2544 scope.go:117] "RemoveContainer" containerID="b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"
Jul 27 20:06:08.152892 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:08.152868    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:06:08.152955 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:08.152933    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:06:08.152993 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:08.152962    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:06:09.158275 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:09.158243    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" event={"ID":"0ba79523-3c6f-4c4d-888a-3c8be5c24862","Type":"ContainerStarted","Data":"af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f"}
Jul 27 20:06:09.997807 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:09.997770    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:09.998013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:09.997986    2544 patch_prober.go:28] interesting pod/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 20:06:09.998058 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:09.998032    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 20:06:10.997674 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:10.997641    2544 patch_prober.go:28] interesting pod/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 20:06:10.998146 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:10.997694    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 20:06:11.997192 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:11.997159    2544 patch_prober.go:28] interesting pod/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 27 20:06:11.997365 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:11.997214    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 27 20:06:13.000694 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:13.000663    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:13.001110 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:13.000927    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:13.001698 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:13.001676    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:21.910874 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:21.910844    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:06:21.926170 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:21.926143    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:06:49.606470 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:49.606425    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"]
Jul 27 20:06:49.607053 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:49.606725    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy" containerID="cri-o://af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f" gracePeriod=30
Jul 27 20:06:54.872152 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.872093    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:54.977009 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.976981    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istiod-ca-cert\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977135 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977027    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-certs\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977135 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977050    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-podinfo\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977135 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977080    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-credential-socket\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977135 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977097    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-socket\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977135 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977117    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-data\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977386 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977141    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-token\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977386 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977173    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqkdc\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-kube-api-access-bqkdc\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977386 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977197    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-envoy\") pod \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\" (UID: \"0ba79523-3c6f-4c4d-888a-3c8be5c24862\") "
Jul 27 20:06:54.977386 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977368    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:54.977560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977388    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:54.977560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977411    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:54.977560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977429    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:06:54.977560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977528    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:54.977560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977545    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:54.977560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977558    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:54.977812 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977571    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:54.978026 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.977991    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:54.979891 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.979865    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:06:54.980130 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.980103    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:06:54.980237 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.980103    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:54.980520 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:54.980504    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-kube-api-access-bqkdc" (OuterVolumeSpecName: "kube-api-access-bqkdc") pod "0ba79523-3c6f-4c4d-888a-3c8be5c24862" (UID: "0ba79523-3c6f-4c4d-888a-3c8be5c24862"). InnerVolumeSpecName "kube-api-access-bqkdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:06:55.077976 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.077952    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:55.077976 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.077972    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:55.078105 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.077982    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqkdc\" (UniqueName: \"kubernetes.io/projected/0ba79523-3c6f-4c4d-888a-3c8be5c24862-kube-api-access-bqkdc\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:55.078105 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.077990    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:55.078105 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.077998    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba79523-3c6f-4c4d-888a-3c8be5c24862-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:06:55.405671 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.405641    2544 generic.go:356] "Generic (PLEG): container finished" podID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerID="af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f" exitCode=0
Jul 27 20:06:55.405844 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.405691    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"
Jul 27 20:06:55.405844 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.405721    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" event={"ID":"0ba79523-3c6f-4c4d-888a-3c8be5c24862","Type":"ContainerDied","Data":"af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f"}
Jul 27 20:06:55.405844 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.405785    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg" event={"ID":"0ba79523-3c6f-4c4d-888a-3c8be5c24862","Type":"ContainerDied","Data":"10aaecbf26bd4051024b48e4a5e727cc728b3963558536ef8a6f521fe75ad39f"}
Jul 27 20:06:55.405844 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.405807    2544 scope.go:117] "RemoveContainer" containerID="af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f"
Jul 27 20:06:55.415283 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.415267    2544 scope.go:117] "RemoveContainer" containerID="b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"
Jul 27 20:06:55.423046 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.423028    2544 scope.go:117] "RemoveContainer" containerID="af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f"
Jul 27 20:06:55.423289 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:06:55.423271    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f\": container with ID starting with af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f not found: ID does not exist" containerID="af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f"
Jul 27 20:06:55.423334 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.423298    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f"} err="failed to get container status \"af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f\": rpc error: code = NotFound desc = could not find container \"af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f\": container with ID starting with af4af1d3c82883cc0ac839f94b8e3244ab4b98b86b961a2cec43aa54e127c18f not found: ID does not exist"
Jul 27 20:06:55.423334 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.423316    2544 scope.go:117] "RemoveContainer" containerID="b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"
Jul 27 20:06:55.423552 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:06:55.423535    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81\": container with ID starting with b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81 not found: ID does not exist" containerID="b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"
Jul 27 20:06:55.423591 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.423558    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81"} err="failed to get container status \"b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81\": rpc error: code = NotFound desc = could not find container \"b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81\": container with ID starting with b950f462336bffb4bc5ba8cdc898fc1216e6c6a18342027b304fb31116adcc81 not found: ID does not exist"
Jul 27 20:06:55.426371 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.426351    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"]
Jul 27 20:06:55.431714 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.431696    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d978f1-openshift-default-9b65d484-j4bgg"]
Jul 27 20:06:55.847100 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:55.847011    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" path="/var/lib/kubelet/pods/0ba79523-3c6f-4c4d-888a-3c8be5c24862/volumes"
Jul 27 20:06:58.947052 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947019    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"]
Jul 27 20:06:58.947623 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947603    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy"
Jul 27 20:06:58.947707 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947626    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy"
Jul 27 20:06:58.947707 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947636    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy"
Jul 27 20:06:58.947707 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947644    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy"
Jul 27 20:06:58.947892 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947783    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy"
Jul 27 20:06:58.947892 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.947799    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ba79523-3c6f-4c4d-888a-3c8be5c24862" containerName="istio-proxy"
Jul 27 20:06:58.950012 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.949994    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:58.952413 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.952395    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-73ce16-openshift-default-dockercfg-pdgjr\""
Jul 27 20:06:58.961329 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:58.961309    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"]
Jul 27 20:06:59.008667 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008639    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-token\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.008816 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008683    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-envoy\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.008816 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008718    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-data\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.008816 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008784    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8mh2\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-kube-api-access-t8mh2\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.008947 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008816    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-certs\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.008947 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008841    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-podinfo\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.008947 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008926    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istiod-ca-cert\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.009042 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008956    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-credential-socket\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.009042 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.008994    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-socket\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110114 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110070    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-token\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110114 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110118    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-envoy\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110145    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-data\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110178    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8mh2\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-kube-api-access-t8mh2\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110200    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-certs\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110216    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-podinfo\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110256    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istiod-ca-cert\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110287    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-credential-socket\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110359 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110331    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-socket\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110698 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110599    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-data\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110698 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110653    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-certs\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.110916 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110888    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-credential-socket\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.111006 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.110992    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-socket\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.111163 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.111146    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istiod-ca-cert\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.112800 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.112780    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-envoy\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.112920 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.112901    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-podinfo\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.118457 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.118432    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-token\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.119044 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.119022    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8mh2\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-kube-api-access-t8mh2\") pod \"e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") " pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.259754 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.259672    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:06:59.398658 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.398620    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"]
Jul 27 20:06:59.400972 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:06:59.400947    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb370f0_a4ef_4708_aa76_2b2037aaad7a.slice/crio-602800a893bfc5b8cbe3dcf16e137942fe0aa0dc295cf0f6a91ccea05f4b5c10 WatchSource:0}: Error finding container 602800a893bfc5b8cbe3dcf16e137942fe0aa0dc295cf0f6a91ccea05f4b5c10: Status 404 returned error can't find the container with id 602800a893bfc5b8cbe3dcf16e137942fe0aa0dc295cf0f6a91ccea05f4b5c10
Jul 27 20:06:59.403180 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.403146    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:06:59.403262 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.403213    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:06:59.403262 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.403241    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:06:59.428468 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:06:59.428437    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" event={"ID":"edb370f0-a4ef-4708-aa76-2b2037aaad7a","Type":"ContainerStarted","Data":"602800a893bfc5b8cbe3dcf16e137942fe0aa0dc295cf0f6a91ccea05f4b5c10"}
Jul 27 20:07:00.434507 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:00.434472    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" event={"ID":"edb370f0-a4ef-4708-aa76-2b2037aaad7a","Type":"ContainerStarted","Data":"2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"}
Jul 27 20:07:00.456480 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:00.456426    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podStartSLOduration=2.4564107760000002 podStartE2EDuration="2.456410776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:07:00.454242746 +0000 UTC m=+2439.111952275" watchObservedRunningTime="2026-07-27 20:07:00.456410776 +0000 UTC m=+2439.114120304"
Jul 27 20:07:01.260409 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:01.260380    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:01.265160 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:01.265139    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:01.443485 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:01.443460    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:01.444339 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:01.444319    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:17.484867 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:17.484834    2544 patch_prober.go:28] interesting pod/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 27 20:07:17.485232 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:17.484885    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 27 20:07:18.530426 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:18.530397    2544 generic.go:356] "Generic (PLEG): container finished" podID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerID="2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81" exitCode=0
Jul 27 20:07:18.530852 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:18.530470    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" event={"ID":"edb370f0-a4ef-4708-aa76-2b2037aaad7a","Type":"ContainerDied","Data":"2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"}
Jul 27 20:07:18.530852 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:18.530819    2544 scope.go:117] "RemoveContainer" containerID="2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"
Jul 27 20:07:18.531777 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:18.531754    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:07:18.531837 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:18.531817    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:07:18.531877 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:18.531848    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:07:19.537324 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:19.537291    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" event={"ID":"edb370f0-a4ef-4708-aa76-2b2037aaad7a","Type":"ContainerStarted","Data":"e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5"}
Jul 27 20:07:20.259906 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:20.259865    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:20.260079 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:20.260049    2544 patch_prober.go:28] interesting pod/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 27 20:07:20.260126 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:20.260098    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 27 20:07:21.260185 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:21.260151    2544 patch_prober.go:28] interesting pod/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 27 20:07:21.260576 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:21.260239    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 27 20:07:22.260345 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:22.260313    2544 patch_prober.go:28] interesting pod/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 27 20:07:22.260685 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:22.260363    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 27 20:07:23.259963 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:23.259930    2544 patch_prober.go:28] interesting pod/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 27 20:07:23.260122 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:23.259981    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 27 20:07:24.263167 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:24.263136    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:24.263613 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:24.263528    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:07:24.264253 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:07:24.264232    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:11:21.944300 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:11:21.944228    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:11:21.959527 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:11:21.959506    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:13:05.927207 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.927177    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"]
Jul 27 20:13:05.930828 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.930808    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:05.933175 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.933150    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"kube-root-ca.crt\""
Jul 27 20:13:05.934369 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.934349    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"openshift-service-ca.crt\""
Jul 27 20:13:05.934494 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.934385    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"default-dockercfg-2jg5m\""
Jul 27 20:13:05.934494 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.934445    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"sub-test-model-73ce16-kserve-self-signed-certs\""
Jul 27 20:13:05.942093 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:05.942049    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"]
Jul 27 20:13:06.012556 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.012534    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-home\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.012665 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.012589    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-model-cache\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.012718 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.012684    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77973f38-ba1e-4564-8c26-1f4266b4f216-tls-certs\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.012777 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.012748    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-dshm\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.012777 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.012770    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tlw7\" (UniqueName: \"kubernetes.io/projected/77973f38-ba1e-4564-8c26-1f4266b4f216-kube-api-access-6tlw7\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.012865 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.012799    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-tmp-dir\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.113957 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.113931    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-dshm\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.113957 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.113965    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tlw7\" (UniqueName: \"kubernetes.io/projected/77973f38-ba1e-4564-8c26-1f4266b4f216-kube-api-access-6tlw7\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114160 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.113996    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-tmp-dir\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114160 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.114014    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-home\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114160 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.114127    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-model-cache\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114300 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.114233    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77973f38-ba1e-4564-8c26-1f4266b4f216-tls-certs\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114383 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.114364    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-tmp-dir\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114440 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.114412    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-home\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.114555 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.114533    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-model-cache\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.116247 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.116223    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-dshm\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.116535 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.116518    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77973f38-ba1e-4564-8c26-1f4266b4f216-tls-certs\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.122290 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.122270    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tlw7\" (UniqueName: \"kubernetes.io/projected/77973f38-ba1e-4564-8c26-1f4266b4f216-kube-api-access-6tlw7\") pod \"sub-test-model-73ce16-kserve-567988b965-cnfhc\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") " pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.242380 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.242327    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:06.373977 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:06.373952    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"]
Jul 27 20:13:06.375703 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:13:06.375675    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77973f38_ba1e_4564_8c26_1f4266b4f216.slice/crio-d33c2497a59ea0c8725c62c38f86f32c672fb4b1f81fb0df002c8068254b4c3b WatchSource:0}: Error finding container d33c2497a59ea0c8725c62c38f86f32c672fb4b1f81fb0df002c8068254b4c3b: Status 404 returned error can't find the container with id d33c2497a59ea0c8725c62c38f86f32c672fb4b1f81fb0df002c8068254b4c3b
Jul 27 20:13:07.305266 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:07.305231    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc" event={"ID":"77973f38-ba1e-4564-8c26-1f4266b4f216","Type":"ContainerStarted","Data":"b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c"}
Jul 27 20:13:07.305704 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:07.305280    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc" event={"ID":"77973f38-ba1e-4564-8c26-1f4266b4f216","Type":"ContainerStarted","Data":"d33c2497a59ea0c8725c62c38f86f32c672fb4b1f81fb0df002c8068254b4c3b"}
Jul 27 20:13:07.322789 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:07.322718    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc" podStartSLOduration=2.3227028020000002 podStartE2EDuration="2.322702802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:13:07.321872945 +0000 UTC m=+2805.979582499" watchObservedRunningTime="2026-07-27 20:13:07.322702802 +0000 UTC m=+2805.980412329"
Jul 27 20:13:16.242499 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:16.242456    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:16.242940 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:16.242515    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:16.254462 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:16.254434    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:13:16.359836 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:13:16.359807    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:14:24.313130 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:24.313060    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"]
Jul 27 20:14:39.331812 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.331765    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc" podUID="77973f38-ba1e-4564-8c26-1f4266b4f216" containerName="main" containerID="cri-o://b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c" gracePeriod=45
Jul 27 20:14:39.586138 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.586092    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:14:39.675822 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.675798    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-home\") pod \"77973f38-ba1e-4564-8c26-1f4266b4f216\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") "
Jul 27 20:14:39.675964 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.675841    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77973f38-ba1e-4564-8c26-1f4266b4f216-tls-certs\") pod \"77973f38-ba1e-4564-8c26-1f4266b4f216\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") "
Jul 27 20:14:39.675964 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.675860    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-dshm\") pod \"77973f38-ba1e-4564-8c26-1f4266b4f216\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") "
Jul 27 20:14:39.675964 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.675901    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-model-cache\") pod \"77973f38-ba1e-4564-8c26-1f4266b4f216\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") "
Jul 27 20:14:39.675964 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.675926    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tlw7\" (UniqueName: \"kubernetes.io/projected/77973f38-ba1e-4564-8c26-1f4266b4f216-kube-api-access-6tlw7\") pod \"77973f38-ba1e-4564-8c26-1f4266b4f216\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") "
Jul 27 20:14:39.676180 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.676014    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-tmp-dir\") pod \"77973f38-ba1e-4564-8c26-1f4266b4f216\" (UID: \"77973f38-ba1e-4564-8c26-1f4266b4f216\") "
Jul 27 20:14:39.676180 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.676034    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-home" (OuterVolumeSpecName: "home") pod "77973f38-ba1e-4564-8c26-1f4266b4f216" (UID: "77973f38-ba1e-4564-8c26-1f4266b4f216"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:39.676279 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.676204    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-model-cache" (OuterVolumeSpecName: "model-cache") pod "77973f38-ba1e-4564-8c26-1f4266b4f216" (UID: "77973f38-ba1e-4564-8c26-1f4266b4f216"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:39.676279 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.676234    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.676353 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.676313    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77973f38-ba1e-4564-8c26-1f4266b4f216" (UID: "77973f38-ba1e-4564-8c26-1f4266b4f216"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:39.677991 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.677967    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-dshm" (OuterVolumeSpecName: "dshm") pod "77973f38-ba1e-4564-8c26-1f4266b4f216" (UID: "77973f38-ba1e-4564-8c26-1f4266b4f216"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:14:39.678122 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.678048    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77973f38-ba1e-4564-8c26-1f4266b4f216-kube-api-access-6tlw7" (OuterVolumeSpecName: "kube-api-access-6tlw7") pod "77973f38-ba1e-4564-8c26-1f4266b4f216" (UID: "77973f38-ba1e-4564-8c26-1f4266b4f216"). InnerVolumeSpecName "kube-api-access-6tlw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:14:39.678122 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.678072    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77973f38-ba1e-4564-8c26-1f4266b4f216-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77973f38-ba1e-4564-8c26-1f4266b4f216" (UID: "77973f38-ba1e-4564-8c26-1f4266b4f216"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:14:39.777558 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.777533    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77973f38-ba1e-4564-8c26-1f4266b4f216-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.777558 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.777554    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.777692 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.777566    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.777692 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.777578    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tlw7\" (UniqueName: \"kubernetes.io/projected/77973f38-ba1e-4564-8c26-1f4266b4f216-kube-api-access-6tlw7\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.777692 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.777616    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77973f38-ba1e-4564-8c26-1f4266b4f216-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:14:39.793991 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.793966    2544 generic.go:356] "Generic (PLEG): container finished" podID="77973f38-ba1e-4564-8c26-1f4266b4f216" containerID="b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c" exitCode=0
Jul 27 20:14:39.794084 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.794015    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"
Jul 27 20:14:39.794084 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.794028    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc" event={"ID":"77973f38-ba1e-4564-8c26-1f4266b4f216","Type":"ContainerDied","Data":"b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c"}
Jul 27 20:14:39.794084 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.794058    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc" event={"ID":"77973f38-ba1e-4564-8c26-1f4266b4f216","Type":"ContainerDied","Data":"d33c2497a59ea0c8725c62c38f86f32c672fb4b1f81fb0df002c8068254b4c3b"}
Jul 27 20:14:39.794084 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.794074    2544 scope.go:117] "RemoveContainer" containerID="b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c"
Jul 27 20:14:39.803182 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.803166    2544 scope.go:117] "RemoveContainer" containerID="b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c"
Jul 27 20:14:39.803445 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:14:39.803424    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c\": container with ID starting with b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c not found: ID does not exist" containerID="b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c"
Jul 27 20:14:39.803487 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.803452    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c"} err="failed to get container status \"b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c\": rpc error: code = NotFound desc = could not find container \"b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c\": container with ID starting with b7ad9a9c68b5a1ee8dfb1fc34bce24b96ca781ebe2d5cfdf8f28111aecbd8e9c not found: ID does not exist"
Jul 27 20:14:39.814526 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.814506    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"]
Jul 27 20:14:39.818357 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.818337    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/sub-test-model-73ce16-kserve-567988b965-cnfhc"]
Jul 27 20:14:39.840638 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:39.840593    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77973f38-ba1e-4564-8c26-1f4266b4f216" path="/var/lib/kubelet/pods/77973f38-ba1e-4564-8c26-1f4266b4f216/volumes"
Jul 27 20:14:48.915410 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.915377    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"]
Jul 27 20:14:48.915812 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.915799    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77973f38-ba1e-4564-8c26-1f4266b4f216" containerName="main"
Jul 27 20:14:48.915812 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.915813    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="77973f38-ba1e-4564-8c26-1f4266b4f216" containerName="main"
Jul 27 20:14:48.915897 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.915877    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="77973f38-ba1e-4564-8c26-1f4266b4f216" containerName="main"
Jul 27 20:14:48.920279 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.920259    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:48.924416 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.924341    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"kube-root-ca.crt\""
Jul 27 20:14:48.924785 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.924544    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"default-dockercfg-2jg5m\""
Jul 27 20:14:48.924934 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.924910    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"rate-test-model-73ce16-kserve-self-signed-certs\""
Jul 27 20:14:48.925133 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.925115    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-73ce16\"/\"openshift-service-ca.crt\""
Jul 27 20:14:48.927264 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:48.927230    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"]
Jul 27 20:14:49.046792 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.046761    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzvns\" (UniqueName: \"kubernetes.io/projected/569ef415-74c1-43d2-8e31-54ec213c7841-kube-api-access-hzvns\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.046930 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.046838    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-dshm\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.046976 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.046930    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-tmp-dir\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.047023 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.046989    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569ef415-74c1-43d2-8e31-54ec213c7841-tls-certs\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.047066 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.047019    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-home\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.047066 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.047053    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-model-cache\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148111 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148085    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569ef415-74c1-43d2-8e31-54ec213c7841-tls-certs\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148281 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148125    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-home\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148281 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148161    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-model-cache\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148281 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148198    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzvns\" (UniqueName: \"kubernetes.io/projected/569ef415-74c1-43d2-8e31-54ec213c7841-kube-api-access-hzvns\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148281 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148256    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-dshm\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148501 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148309    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-tmp-dir\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148587 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148562    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-home\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148647 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148591    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-model-cache\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.148647 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.148625    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-tmp-dir\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.150226 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.150202    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-dshm\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.150386 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.150370    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569ef415-74c1-43d2-8e31-54ec213c7841-tls-certs\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.156597 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.156575    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzvns\" (UniqueName: \"kubernetes.io/projected/569ef415-74c1-43d2-8e31-54ec213c7841-kube-api-access-hzvns\") pod \"rate-test-model-73ce16-kserve-558f4fbb79-n2r9f\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") " pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.231670 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.231624    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:49.363356 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.363329    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"]
Jul 27 20:14:49.365069 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:14:49.365042    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod569ef415_74c1_43d2_8e31_54ec213c7841.slice/crio-c6220b3c6c7e384d10b5c4a83774ab2a07dd60844be953b63a754b9922d6a853 WatchSource:0}: Error finding container c6220b3c6c7e384d10b5c4a83774ab2a07dd60844be953b63a754b9922d6a853: Status 404 returned error can't find the container with id c6220b3c6c7e384d10b5c4a83774ab2a07dd60844be953b63a754b9922d6a853
Jul 27 20:14:49.844203 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.844172    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f" event={"ID":"569ef415-74c1-43d2-8e31-54ec213c7841","Type":"ContainerStarted","Data":"0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948"}
Jul 27 20:14:49.844203 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.844207    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f" event={"ID":"569ef415-74c1-43d2-8e31-54ec213c7841","Type":"ContainerStarted","Data":"c6220b3c6c7e384d10b5c4a83774ab2a07dd60844be953b63a754b9922d6a853"}
Jul 27 20:14:49.862234 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:49.862188    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f" podStartSLOduration=1.862173398 podStartE2EDuration="1.862173398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:14:49.860561213 +0000 UTC m=+2908.518270741" watchObservedRunningTime="2026-07-27 20:14:49.862173398 +0000 UTC m=+2908.519882966"
Jul 27 20:14:59.232570 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:59.232537    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:59.233098 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:59.232626    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:59.244851 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:59.244827    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:14:59.906887 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:14:59.906862    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:15:57.805468 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:15:57.805438    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"]
Jul 27 20:16:12.823580 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:12.823541    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f" podUID="569ef415-74c1-43d2-8e31-54ec213c7841" containerName="main" containerID="cri-o://0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948" gracePeriod=45
Jul 27 20:16:13.077021 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.076974    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:16:13.207694 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207669    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzvns\" (UniqueName: \"kubernetes.io/projected/569ef415-74c1-43d2-8e31-54ec213c7841-kube-api-access-hzvns\") pod \"569ef415-74c1-43d2-8e31-54ec213c7841\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") "
Jul 27 20:16:13.207847 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207700    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-model-cache\") pod \"569ef415-74c1-43d2-8e31-54ec213c7841\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") "
Jul 27 20:16:13.207847 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207725    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-tmp-dir\") pod \"569ef415-74c1-43d2-8e31-54ec213c7841\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") "
Jul 27 20:16:13.207968 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207873    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569ef415-74c1-43d2-8e31-54ec213c7841-tls-certs\") pod \"569ef415-74c1-43d2-8e31-54ec213c7841\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") "
Jul 27 20:16:13.207968 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207920    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-home\") pod \"569ef415-74c1-43d2-8e31-54ec213c7841\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") "
Jul 27 20:16:13.207968 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207957    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "569ef415-74c1-43d2-8e31-54ec213c7841" (UID: "569ef415-74c1-43d2-8e31-54ec213c7841"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:13.208109 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207976    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-dshm\") pod \"569ef415-74c1-43d2-8e31-54ec213c7841\" (UID: \"569ef415-74c1-43d2-8e31-54ec213c7841\") "
Jul 27 20:16:13.208109 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.207979    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-model-cache" (OuterVolumeSpecName: "model-cache") pod "569ef415-74c1-43d2-8e31-54ec213c7841" (UID: "569ef415-74c1-43d2-8e31-54ec213c7841"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:13.208181 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.208129    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-home" (OuterVolumeSpecName: "home") pod "569ef415-74c1-43d2-8e31-54ec213c7841" (UID: "569ef415-74c1-43d2-8e31-54ec213c7841"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:13.208314 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.208294    2544 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.208413 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.208316    2544 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.208413 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.208328    2544 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.209737 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.209711    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/569ef415-74c1-43d2-8e31-54ec213c7841-kube-api-access-hzvns" (OuterVolumeSpecName: "kube-api-access-hzvns") pod "569ef415-74c1-43d2-8e31-54ec213c7841" (UID: "569ef415-74c1-43d2-8e31-54ec213c7841"). InnerVolumeSpecName "kube-api-access-hzvns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:16:13.209847 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.209832    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/569ef415-74c1-43d2-8e31-54ec213c7841-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "569ef415-74c1-43d2-8e31-54ec213c7841" (UID: "569ef415-74c1-43d2-8e31-54ec213c7841"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 20:16:13.209899 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.209885    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-dshm" (OuterVolumeSpecName: "dshm") pod "569ef415-74c1-43d2-8e31-54ec213c7841" (UID: "569ef415-74c1-43d2-8e31-54ec213c7841"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:16:13.267980 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.267955    2544 generic.go:356] "Generic (PLEG): container finished" podID="569ef415-74c1-43d2-8e31-54ec213c7841" containerID="0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948" exitCode=0
Jul 27 20:16:13.268058 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.267996    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"
Jul 27 20:16:13.268058 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.268041    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f" event={"ID":"569ef415-74c1-43d2-8e31-54ec213c7841","Type":"ContainerDied","Data":"0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948"}
Jul 27 20:16:13.268138 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.268076    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f" event={"ID":"569ef415-74c1-43d2-8e31-54ec213c7841","Type":"ContainerDied","Data":"c6220b3c6c7e384d10b5c4a83774ab2a07dd60844be953b63a754b9922d6a853"}
Jul 27 20:16:13.268138 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.268094    2544 scope.go:117] "RemoveContainer" containerID="0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948"
Jul 27 20:16:13.277543 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.277525    2544 scope.go:117] "RemoveContainer" containerID="0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948"
Jul 27 20:16:13.277786 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:16:13.277770    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948\": container with ID starting with 0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948 not found: ID does not exist" containerID="0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948"
Jul 27 20:16:13.277835 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.277794    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948"} err="failed to get container status \"0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948\": rpc error: code = NotFound desc = could not find container \"0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948\": container with ID starting with 0b1827f10abd8404d30ba0614e8f350ba837028fd82901bd4f9387af3f1d7948 not found: ID does not exist"
Jul 27 20:16:13.290840 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.290816    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"]
Jul 27 20:16:13.297341 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.297322    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-73ce16/rate-test-model-73ce16-kserve-558f4fbb79-n2r9f"]
Jul 27 20:16:13.308707 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.308685    2544 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/569ef415-74c1-43d2-8e31-54ec213c7841-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.308817 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.308759    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzvns\" (UniqueName: \"kubernetes.io/projected/569ef415-74c1-43d2-8e31-54ec213c7841-kube-api-access-hzvns\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.308817 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.308776    2544 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/569ef415-74c1-43d2-8e31-54ec213c7841-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:16:13.840884 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:13.840860    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="569ef415-74c1-43d2-8e31-54ec213c7841" path="/var/lib/kubelet/pods/569ef415-74c1-43d2-8e31-54ec213c7841/volumes"
Jul 27 20:16:17.465291 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.465262    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"]
Jul 27 20:16:17.465717 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.465688    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569ef415-74c1-43d2-8e31-54ec213c7841" containerName="main"
Jul 27 20:16:17.465717 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.465701    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="569ef415-74c1-43d2-8e31-54ec213c7841" containerName="main"
Jul 27 20:16:17.465898 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.465835    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="569ef415-74c1-43d2-8e31-54ec213c7841" containerName="main"
Jul 27 20:16:17.470376 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.470355    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.472864 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.472844    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-018406-openshift-default-dockercfg-6cbdw\""
Jul 27 20:16:17.480508 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.480470    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"]
Jul 27 20:16:17.645046 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645009    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-socket\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645232 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645049    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-data\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645232 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645094    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-envoy\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645232 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645115    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-certs\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645232 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645142    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-podinfo\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645232 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645221    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-token\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645464 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645278    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3832311-bcdb-4f1e-9165-3330f7072cda-istiod-ca-cert\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645464 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645339    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4bq4\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-kube-api-access-c4bq4\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.645464 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.645374    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-credential-socket\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746343 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746277    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-token\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746343 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746314    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3832311-bcdb-4f1e-9165-3330f7072cda-istiod-ca-cert\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746528 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746347    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4bq4\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-kube-api-access-c4bq4\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746528 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746372    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-credential-socket\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746528 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746396    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-socket\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746528 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746412    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-data\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746528 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746438    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-envoy\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746528 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746461    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-certs\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746871 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746570    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-podinfo\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746871 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746794    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-credential-socket\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.746871 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746858    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-certs\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.747018 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746896    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-socket\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.747018 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.746955    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-data\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.747358 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.747338    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3832311-bcdb-4f1e-9165-3330f7072cda-istiod-ca-cert\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.748741 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.748702    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-envoy\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.748939 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.748922    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-podinfo\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.753832 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.753811    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-token\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.754016 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.753999    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4bq4\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-kube-api-access-c4bq4\") pod \"e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") " pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.781902 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.781886    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:17.919629 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.919607    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"]
Jul 27 20:16:17.921693 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:16:17.921666    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3832311_bcdb_4f1e_9165_3330f7072cda.slice/crio-599d37861c48cd9d2187f41fff67b56023d1eecda855fa0513365dca4292e0c4 WatchSource:0}: Error finding container 599d37861c48cd9d2187f41fff67b56023d1eecda855fa0513365dca4292e0c4: Status 404 returned error can't find the container with id 599d37861c48cd9d2187f41fff67b56023d1eecda855fa0513365dca4292e0c4
Jul 27 20:16:17.924155 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.924122    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:16:17.924257 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.924202    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:16:17.924257 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:17.924245    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:16:18.294781 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:18.294746    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" event={"ID":"d3832311-bcdb-4f1e-9165-3330f7072cda","Type":"ContainerStarted","Data":"a75feedbe90bc01cbab88011c49a73ce7d456bd2c79a85b1dd73ee79a092dc01"}
Jul 27 20:16:18.294947 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:18.294798    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" event={"ID":"d3832311-bcdb-4f1e-9165-3330f7072cda","Type":"ContainerStarted","Data":"599d37861c48cd9d2187f41fff67b56023d1eecda855fa0513365dca4292e0c4"}
Jul 27 20:16:18.313314 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:18.313272    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podStartSLOduration=1.313257141 podStartE2EDuration="1.313257141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:16:18.312349979 +0000 UTC m=+2996.970059509" watchObservedRunningTime="2026-07-27 20:16:18.313257141 +0000 UTC m=+2996.970966663"
Jul 27 20:16:18.782537 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:18.782510    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:18.787221 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:18.787200    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:19.299783 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:19.299749    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:19.300724 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:19.300707    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:21.977330 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:21.977295    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:16:21.992090 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:21.992068    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:16:38.413618 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:38.413583    2544 generic.go:356] "Generic (PLEG): container finished" podID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerID="a75feedbe90bc01cbab88011c49a73ce7d456bd2c79a85b1dd73ee79a092dc01" exitCode=0
Jul 27 20:16:38.414094 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:38.413661    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" event={"ID":"d3832311-bcdb-4f1e-9165-3330f7072cda","Type":"ContainerDied","Data":"a75feedbe90bc01cbab88011c49a73ce7d456bd2c79a85b1dd73ee79a092dc01"}
Jul 27 20:16:38.414094 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:38.414034    2544 scope.go:117] "RemoveContainer" containerID="a75feedbe90bc01cbab88011c49a73ce7d456bd2c79a85b1dd73ee79a092dc01"
Jul 27 20:16:38.415153 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:38.415119    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:16:38.415276 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:38.415208    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:16:38.415276 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:38.415258    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:16:39.421404 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:39.421371    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" event={"ID":"d3832311-bcdb-4f1e-9165-3330f7072cda","Type":"ContainerStarted","Data":"38e49725dee9b6c9d5009bef187528b3d6ae4471c33a300dab7a294418926cec"}
Jul 27 20:16:39.782561 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:39.782465    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:39.782750 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:39.782704    2544 patch_prober.go:28] interesting pod/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 27 20:16:39.782835 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:39.782791    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 27 20:16:40.783114 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:40.783073    2544 patch_prober.go:28] interesting pod/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 27 20:16:40.783580 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:40.783145    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 27 20:16:41.782852 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:41.782817    2544 patch_prober.go:28] interesting pod/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 27 20:16:41.783048 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:41.782888    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 27 20:16:42.783184 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:42.783133    2544 patch_prober.go:28] interesting pod/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 27 20:16:42.783630 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:42.783206    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 27 20:16:43.786225 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:43.786197    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:43.786614 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:43.786596    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:16:43.787241 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:16:43.787223    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:17:00.089916 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:00.089881    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gbhk5_602f0fdc-6d53-45ad-a048-112e95eb5f7b/payload-processing/0.log"
Jul 27 20:17:50.050843 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.050769    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gbhk5_602f0fdc-6d53-45ad-a048-112e95eb5f7b/payload-processing/0.log"
Jul 27 20:17:50.694647 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.694607    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"]
Jul 27 20:17:50.698303 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.698282    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.700692 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.700672    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-394c6a-openshift-default-dockercfg-wn9cp\""
Jul 27 20:17:50.709968 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.709948    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"]
Jul 27 20:17:50.805406 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805380    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-istio-token\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805537 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805412    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4669b11a-4584-4d81-9328-6c4035af7020-istio-podinfo\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805537 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805439    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-envoy\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805630 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805543    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7khf\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-kube-api-access-f7khf\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805630 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805577    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-certs\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805630 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805607    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4669b11a-4584-4d81-9328-6c4035af7020-istiod-ca-cert\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805836 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805688    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-socket\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805836 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805771    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-data\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.805836 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.805813    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-credential-socket\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907090 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907059    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-credential-socket\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907261 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907113    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-istio-token\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907261 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907230    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4669b11a-4584-4d81-9328-6c4035af7020-istio-podinfo\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907396 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907300    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-envoy\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907396 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907344    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7khf\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-kube-api-access-f7khf\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907396 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907385    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-certs\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907552 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907416    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-credential-socket\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907552 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907434    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4669b11a-4584-4d81-9328-6c4035af7020-istiod-ca-cert\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907552 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907479    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-socket\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907552 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907520    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-data\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907774 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907753    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-certs\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.907864 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907842    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-socket\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.908010 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.907990    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-data\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.908058 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.908042    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4669b11a-4584-4d81-9328-6c4035af7020-istiod-ca-cert\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.909480 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.909459    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-envoy\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.909607 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.909586    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4669b11a-4584-4d81-9328-6c4035af7020-istio-podinfo\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.914298 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.914278    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-istio-token\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:50.914489 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:50.914473    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7khf\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-kube-api-access-f7khf\") pod \"e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") " pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:51.010015 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.009967    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:51.144081 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.144057    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"]
Jul 27 20:17:51.145926 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:17:51.145897    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4669b11a_4584_4d81_9328_6c4035af7020.slice/crio-10f93e5e366119f8024426fc58c0af98ea147b2aebbd3b6d6f638716761f3f12 WatchSource:0}: Error finding container 10f93e5e366119f8024426fc58c0af98ea147b2aebbd3b6d6f638716761f3f12: Status 404 returned error can't find the container with id 10f93e5e366119f8024426fc58c0af98ea147b2aebbd3b6d6f638716761f3f12
Jul 27 20:17:51.148357 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.148327    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:17:51.148437 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.148387    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:17:51.148437 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.148413    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:17:51.782109 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.782076    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" event={"ID":"4669b11a-4584-4d81-9328-6c4035af7020","Type":"ContainerStarted","Data":"686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"}
Jul 27 20:17:51.782262 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.782128    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" event={"ID":"4669b11a-4584-4d81-9328-6c4035af7020","Type":"ContainerStarted","Data":"10f93e5e366119f8024426fc58c0af98ea147b2aebbd3b6d6f638716761f3f12"}
Jul 27 20:17:51.802514 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:51.802459    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podStartSLOduration=1.80243989 podStartE2EDuration="1.80243989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:17:51.800633773 +0000 UTC m=+3090.458343302" watchObservedRunningTime="2026-07-27 20:17:51.80243989 +0000 UTC m=+3090.460149418"
Jul 27 20:17:52.010794 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:52.010764    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:52.015952 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:52.015929    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:52.786717 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:52.786692    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:17:52.787587 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:17:52.787569    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:06.858242 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:06.858209    2544 generic.go:356] "Generic (PLEG): container finished" podID="4669b11a-4584-4d81-9328-6c4035af7020" containerID="686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308" exitCode=0
Jul 27 20:18:06.858619 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:06.858285    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" event={"ID":"4669b11a-4584-4d81-9328-6c4035af7020","Type":"ContainerDied","Data":"686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"}
Jul 27 20:18:06.858683 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:06.858664    2544 scope.go:117] "RemoveContainer" containerID="686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"
Jul 27 20:18:06.859796 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:06.859767    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:18:06.859866 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:06.859845    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:18:06.859907 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:06.859875    2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 27 20:18:07.786856 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:07.786819    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:07.866160 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:07.866125    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" event={"ID":"4669b11a-4584-4d81-9328-6c4035af7020","Type":"ContainerStarted","Data":"8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62"}
Jul 27 20:18:08.011014 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:08.010983    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:08.012263 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:08.012231    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 20:18:08.012398 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:08.012276    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 20:18:09.010762 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.010711    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 20:18:09.011201 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.010782    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 20:18:09.040083 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.040053    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"]
Jul 27 20:18:09.042830 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.042815    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:09.045061 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.045038    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-875sf\""
Jul 27 20:18:09.050387 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.050366    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"]
Jul 27 20:18:09.175200 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.175170    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/515e57ac-278d-4de1-b33d-46bfb0d4e9fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg\" (UID: \"515e57ac-278d-4de1-b33d-46bfb0d4e9fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:09.276255 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.276176    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/515e57ac-278d-4de1-b33d-46bfb0d4e9fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg\" (UID: \"515e57ac-278d-4de1-b33d-46bfb0d4e9fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:09.277262 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.277239    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/515e57ac-278d-4de1-b33d-46bfb0d4e9fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg\" (UID: \"515e57ac-278d-4de1-b33d-46bfb0d4e9fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:09.354184 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.354168    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:09.532290 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.532262    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"]
Jul 27 20:18:09.533900 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:18:09.533876    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod515e57ac_278d_4de1_b33d_46bfb0d4e9fc.slice/crio-ff07845feb52b55c3d8d43ee1f4b7951f4c6ac8c77e4630db4d11c0d47adb54f WatchSource:0}: Error finding container ff07845feb52b55c3d8d43ee1f4b7951f4c6ac8c77e4630db4d11c0d47adb54f: Status 404 returned error can't find the container with id ff07845feb52b55c3d8d43ee1f4b7951f4c6ac8c77e4630db4d11c0d47adb54f
Jul 27 20:18:09.535967 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.535953    2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:18:09.887419 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:09.887021    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg" event={"ID":"515e57ac-278d-4de1-b33d-46bfb0d4e9fc","Type":"ContainerStarted","Data":"ff07845feb52b55c3d8d43ee1f4b7951f4c6ac8c77e4630db4d11c0d47adb54f"}
Jul 27 20:18:10.010701 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:10.010671    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 20:18:10.010889 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:10.010727    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 20:18:11.010805 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:11.010660    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 27 20:18:11.010805 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:11.010748    2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 27 20:18:12.014788 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:12.014757    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:12.015262 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:12.015244    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:12.015829 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:12.015805    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:12.907481 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:12.907451    2544 generic.go:356] "Generic (PLEG): container finished" podID="515e57ac-278d-4de1-b33d-46bfb0d4e9fc" containerID="9d23669a7b16dfdfd5a706f7f2b67f91bf6b51b246e479fc401869dad5df16c9" exitCode=0
Jul 27 20:18:12.907587 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:12.907535    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg" event={"ID":"515e57ac-278d-4de1-b33d-46bfb0d4e9fc","Type":"ContainerDied","Data":"9d23669a7b16dfdfd5a706f7f2b67f91bf6b51b246e479fc401869dad5df16c9"}
Jul 27 20:18:14.049303 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.049280    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:14.232699 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.232642    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/515e57ac-278d-4de1-b33d-46bfb0d4e9fc-maas-api-service-ca\") pod \"515e57ac-278d-4de1-b33d-46bfb0d4e9fc\" (UID: \"515e57ac-278d-4de1-b33d-46bfb0d4e9fc\") "
Jul 27 20:18:14.233037 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.233014    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/515e57ac-278d-4de1-b33d-46bfb0d4e9fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "515e57ac-278d-4de1-b33d-46bfb0d4e9fc" (UID: "515e57ac-278d-4de1-b33d-46bfb0d4e9fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:18:14.333905 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.333876    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/515e57ac-278d-4de1-b33d-46bfb0d4e9fc-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:14.919703 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.919673    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"
Jul 27 20:18:14.919919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.919716    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg" event={"ID":"515e57ac-278d-4de1-b33d-46bfb0d4e9fc","Type":"ContainerDied","Data":"ff07845feb52b55c3d8d43ee1f4b7951f4c6ac8c77e4630db4d11c0d47adb54f"}
Jul 27 20:18:14.919919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:14.919790    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff07845feb52b55c3d8d43ee1f4b7951f4c6ac8c77e4630db4d11c0d47adb54f"
Jul 27 20:18:19.626775 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:19.626686    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"]
Jul 27 20:18:19.629049 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:19.629026    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-394c6a-wvzzg"]
Jul 27 20:18:19.842720 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:19.842690    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="515e57ac-278d-4de1-b33d-46bfb0d4e9fc" path="/var/lib/kubelet/pods/515e57ac-278d-4de1-b33d-46bfb0d4e9fc/volumes"
Jul 27 20:18:26.226342 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:26.226310    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"]
Jul 27 20:18:26.227270 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:26.226577    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy" containerID="cri-o://8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62" gracePeriod=30
Jul 27 20:18:27.017264 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:27.017229    2544 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 20:18:27.017458 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:27.017299    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 20:18:31.487288 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.487266    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:31.571801 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571743    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-envoy\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.571919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571825    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-certs\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.571919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571845    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7khf\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-kube-api-access-f7khf\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.571919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571861    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-credential-socket\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.571919 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571900    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-istio-token\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.572127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571922    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-socket\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.572127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571945    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4669b11a-4584-4d81-9328-6c4035af7020-istiod-ca-cert\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.572127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.571975    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4669b11a-4584-4d81-9328-6c4035af7020-istio-podinfo\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.572127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572002    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-data\") pod \"4669b11a-4584-4d81-9328-6c4035af7020\" (UID: \"4669b11a-4584-4d81-9328-6c4035af7020\") "
Jul 27 20:18:31.572307 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572120    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:31.572307 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572159    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:31.572484 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572377    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4669b11a-4584-4d81-9328-6c4035af7020-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:18:31.572484 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572397    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.572484 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572417    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.572484 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.572429    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:31.573073 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.573049    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-data" (OuterVolumeSpecName: "istio-data") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:31.574073 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.574052    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-istio-token" (OuterVolumeSpecName: "istio-token") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:31.574137 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.574102    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:18:31.574172 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.574161    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4669b11a-4584-4d81-9328-6c4035af7020-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:18:31.574234 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.574219    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-kube-api-access-f7khf" (OuterVolumeSpecName: "kube-api-access-f7khf") pod "4669b11a-4584-4d81-9328-6c4035af7020" (UID: "4669b11a-4584-4d81-9328-6c4035af7020"). InnerVolumeSpecName "kube-api-access-f7khf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:18:31.673840 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673813    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.673840 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673838    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.673982 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673851    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4669b11a-4584-4d81-9328-6c4035af7020-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.673982 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673862    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4669b11a-4584-4d81-9328-6c4035af7020-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.673982 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673873    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.673982 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673883    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4669b11a-4584-4d81-9328-6c4035af7020-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:31.673982 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:31.673894    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7khf\" (UniqueName: \"kubernetes.io/projected/4669b11a-4584-4d81-9328-6c4035af7020-kube-api-access-f7khf\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:18:32.016371 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.016339    2544 generic.go:356] "Generic (PLEG): container finished" podID="4669b11a-4584-4d81-9328-6c4035af7020" containerID="8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62" exitCode=0
Jul 27 20:18:32.016508 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.016382    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"
Jul 27 20:18:32.016508 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.016426    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" event={"ID":"4669b11a-4584-4d81-9328-6c4035af7020","Type":"ContainerDied","Data":"8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62"}
Jul 27 20:18:32.016508 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.016496    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m" event={"ID":"4669b11a-4584-4d81-9328-6c4035af7020","Type":"ContainerDied","Data":"10f93e5e366119f8024426fc58c0af98ea147b2aebbd3b6d6f638716761f3f12"}
Jul 27 20:18:32.016632 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.016525    2544 scope.go:117] "RemoveContainer" containerID="8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62"
Jul 27 20:18:32.025469 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.025452    2544 scope.go:117] "RemoveContainer" containerID="686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"
Jul 27 20:18:32.034655 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.034636    2544 scope.go:117] "RemoveContainer" containerID="8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62"
Jul 27 20:18:32.035168 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:18:32.035147    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62\": container with ID starting with 8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62 not found: ID does not exist" containerID="8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62"
Jul 27 20:18:32.035267 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.035174    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62"} err="failed to get container status \"8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62\": rpc error: code = NotFound desc = could not find container \"8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62\": container with ID starting with 8ab21eb1d874946e77d6f02c4fac0a1973d06bf143e7dc1713cfe82ee65f7e62 not found: ID does not exist"
Jul 27 20:18:32.035267 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.035195    2544 scope.go:117] "RemoveContainer" containerID="686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"
Jul 27 20:18:32.035498 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:18:32.035471    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308\": container with ID starting with 686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308 not found: ID does not exist" containerID="686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"
Jul 27 20:18:32.035616 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.035514    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308"} err="failed to get container status \"686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308\": rpc error: code = NotFound desc = could not find container \"686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308\": container with ID starting with 686dd2d3120e619d9cdb5803fdae9dee90f9deb34aa4e736a25107cb972a1308 not found: ID does not exist"
Jul 27 20:18:32.037025 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.037007    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"]
Jul 27 20:18:32.041878 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:32.041860    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-394c6a-openshift-default-7cf5b4f69b-5529m"]
Jul 27 20:18:33.841138 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:18:33.841099    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4669b11a-4584-4d81-9328-6c4035af7020" path="/var/lib/kubelet/pods/4669b11a-4584-4d81-9328-6c4035af7020/volumes"
Jul 27 20:19:40.168560 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.168528    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"]
Jul 27 20:19:40.169117 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.168780    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy" containerID="cri-o://38e49725dee9b6c9d5009bef187528b3d6ae4471c33a300dab7a294418926cec" gracePeriod=30
Jul 27 20:19:40.549466 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.549377    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"]
Jul 27 20:19:40.549901 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.549883    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy"
Jul 27 20:19:40.550012 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.549903    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy"
Jul 27 20:19:40.550012 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.549915    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="515e57ac-278d-4de1-b33d-46bfb0d4e9fc" containerName="revoke-keys"
Jul 27 20:19:40.550012 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.549923    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="515e57ac-278d-4de1-b33d-46bfb0d4e9fc" containerName="revoke-keys"
Jul 27 20:19:40.550165 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.550029    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="515e57ac-278d-4de1-b33d-46bfb0d4e9fc" containerName="revoke-keys"
Jul 27 20:19:40.550165 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.550043    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy"
Jul 27 20:19:40.550165 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.550055    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy"
Jul 27 20:19:40.553293 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.553274    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:40.555857 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.555837    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-875sf\""
Jul 27 20:19:40.560916 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.560715    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"]
Jul 27 20:19:40.627171 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.627145    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa59718-3936-46a0-8a14-6002b9a442bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5\" (UID: \"5aa59718-3936-46a0-8a14-6002b9a442bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:40.727980 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.727957    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa59718-3936-46a0-8a14-6002b9a442bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5\" (UID: \"5aa59718-3936-46a0-8a14-6002b9a442bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:40.728562 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.728545    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa59718-3936-46a0-8a14-6002b9a442bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5\" (UID: \"5aa59718-3936-46a0-8a14-6002b9a442bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:40.865603 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:40.865583    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:41.202258 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:41.202236    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"]
Jul 27 20:19:41.204547 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:19:41.204525    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa59718_3936_46a0_8a14_6002b9a442bc.slice/crio-25a2aaf707abcf4be4a1522f6fa4a86ac9af78dbbab83f9a5138090c81e493e3 WatchSource:0}: Error finding container 25a2aaf707abcf4be4a1522f6fa4a86ac9af78dbbab83f9a5138090c81e493e3: Status 404 returned error can't find the container with id 25a2aaf707abcf4be4a1522f6fa4a86ac9af78dbbab83f9a5138090c81e493e3
Jul 27 20:19:41.373745 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:41.373702    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5" event={"ID":"5aa59718-3936-46a0-8a14-6002b9a442bc","Type":"ContainerStarted","Data":"25a2aaf707abcf4be4a1522f6fa4a86ac9af78dbbab83f9a5138090c81e493e3"}
Jul 27 20:19:42.381383 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:42.381354    2544 generic.go:356] "Generic (PLEG): container finished" podID="5aa59718-3936-46a0-8a14-6002b9a442bc" containerID="eb6247cb14483df0f6cf3aea4c6313b6dbedf1d877fb9a46dc6f0d0b4109a975" exitCode=0
Jul 27 20:19:42.381849 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:42.381394    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5" event={"ID":"5aa59718-3936-46a0-8a14-6002b9a442bc","Type":"ContainerDied","Data":"eb6247cb14483df0f6cf3aea4c6313b6dbedf1d877fb9a46dc6f0d0b4109a975"}
Jul 27 20:19:43.510750 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:43.510712    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:43.553063 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:43.553036    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa59718-3936-46a0-8a14-6002b9a442bc-maas-api-service-ca\") pod \"5aa59718-3936-46a0-8a14-6002b9a442bc\" (UID: \"5aa59718-3936-46a0-8a14-6002b9a442bc\") "
Jul 27 20:19:43.553361 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:43.553338    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5aa59718-3936-46a0-8a14-6002b9a442bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5aa59718-3936-46a0-8a14-6002b9a442bc" (UID: "5aa59718-3936-46a0-8a14-6002b9a442bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:19:43.654547 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:43.654492    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5aa59718-3936-46a0-8a14-6002b9a442bc-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:43.788929 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:43.788903    2544 patch_prober.go:28] interesting pod/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 27 20:19:43.789038 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:43.788949    2544 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 27 20:19:44.392473 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:44.392452    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"
Jul 27 20:19:44.392630 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:44.392474    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5" event={"ID":"5aa59718-3936-46a0-8a14-6002b9a442bc","Type":"ContainerDied","Data":"25a2aaf707abcf4be4a1522f6fa4a86ac9af78dbbab83f9a5138090c81e493e3"}
Jul 27 20:19:44.392630 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:44.392519    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25a2aaf707abcf4be4a1522f6fa4a86ac9af78dbbab83f9a5138090c81e493e3"
Jul 27 20:19:45.401005 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.400980    2544 generic.go:356] "Generic (PLEG): container finished" podID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerID="38e49725dee9b6c9d5009bef187528b3d6ae4471c33a300dab7a294418926cec" exitCode=0
Jul 27 20:19:45.401378 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.401048    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" event={"ID":"d3832311-bcdb-4f1e-9165-3330f7072cda","Type":"ContainerDied","Data":"38e49725dee9b6c9d5009bef187528b3d6ae4471c33a300dab7a294418926cec"}
Jul 27 20:19:45.401378 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.401106    2544 scope.go:117] "RemoveContainer" containerID="a75feedbe90bc01cbab88011c49a73ce7d456bd2c79a85b1dd73ee79a092dc01"
Jul 27 20:19:45.433420 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.433375    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:19:45.466019 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.465999    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4bq4\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-kube-api-access-c4bq4\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466112 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466043    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-certs\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466112 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466071    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-podinfo\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466188 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466134    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-envoy\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466188 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466167    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-socket\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466280 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466210    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-data\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466280 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466250    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-token\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466384 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466283    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3832311-bcdb-4f1e-9165-3330f7072cda-istiod-ca-cert\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466384 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466324    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-credential-socket\") pod \"d3832311-bcdb-4f1e-9165-3330f7072cda\" (UID: \"d3832311-bcdb-4f1e-9165-3330f7072cda\") "
Jul 27 20:19:45.466384 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466329    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:45.466536 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466524    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:45.466778 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466759    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.466954 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466933    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.466954 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.466836    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3832311-bcdb-4f1e-9165-3330f7072cda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:19:45.467118 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.467052    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:45.467382 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.467350    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:45.468279 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.468247    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:19:45.468395 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.468373    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:19:45.468628 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.468609    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:19:45.468765 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.468646    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-kube-api-access-c4bq4" (OuterVolumeSpecName: "kube-api-access-c4bq4") pod "d3832311-bcdb-4f1e-9165-3330f7072cda" (UID: "d3832311-bcdb-4f1e-9165-3330f7072cda"). InnerVolumeSpecName "kube-api-access-c4bq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:19:45.567529 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567499    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4bq4\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-kube-api-access-c4bq4\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.567529 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567529    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.567529 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567538    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.567688 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567546    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.567688 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567554    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.567688 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567561    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3832311-bcdb-4f1e-9165-3330f7072cda-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:45.567688 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:45.567569    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3832311-bcdb-4f1e-9165-3330f7072cda-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:19:46.407633 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:46.407609    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"
Jul 27 20:19:46.408024 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:46.407639    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh" event={"ID":"d3832311-bcdb-4f1e-9165-3330f7072cda","Type":"ContainerDied","Data":"599d37861c48cd9d2187f41fff67b56023d1eecda855fa0513365dca4292e0c4"}
Jul 27 20:19:46.408024 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:46.407690    2544 scope.go:117] "RemoveContainer" containerID="38e49725dee9b6c9d5009bef187528b3d6ae4471c33a300dab7a294418926cec"
Jul 27 20:19:46.426379 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:46.426359    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"]
Jul 27 20:19:46.432019 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:46.431997    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-018406-openshift-default-57cd545f48-6t5wh"]
Jul 27 20:19:47.842278 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:47.842246    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" path="/var/lib/kubelet/pods/d3832311-bcdb-4f1e-9165-3330f7072cda/volumes"
Jul 27 20:19:51.130806 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:51.130760    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"]
Jul 27 20:19:51.132621 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:51.132596    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-933254-hjxb5"]
Jul 27 20:19:51.842257 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:51.842226    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aa59718-3936-46a0-8a14-6002b9a442bc" path="/var/lib/kubelet/pods/5aa59718-3936-46a0-8a14-6002b9a442bc/volumes"
Jul 27 20:19:58.193581 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193545    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxjl4"]
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193942    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5aa59718-3936-46a0-8a14-6002b9a442bc" containerName="revoke-keys"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193952    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="5aa59718-3936-46a0-8a14-6002b9a442bc" containerName="revoke-keys"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193971    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193977    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="4669b11a-4584-4d81-9328-6c4035af7020" containerName="istio-proxy"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193983    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193988    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193994    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy"
Jul 27 20:19:58.194013 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.193999    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy"
Jul 27 20:19:58.194404 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.194057    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy"
Jul 27 20:19:58.194404 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.194067    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3832311-bcdb-4f1e-9165-3330f7072cda" containerName="istio-proxy"
Jul 27 20:19:58.194404 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.194076    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="5aa59718-3936-46a0-8a14-6002b9a442bc" containerName="revoke-keys"
Jul 27 20:19:58.198304 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.198284    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:19:58.205093 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.205069    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxjl4"]
Jul 27 20:19:58.285133 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.285102    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjb7g\" (UniqueName: \"kubernetes.io/projected/28476f03-603f-44f5-8251-841fb48941f0-kube-api-access-kjb7g\") pod \"kuadrant-operator-catalog-vxjl4\" (UID: \"28476f03-603f-44f5-8251-841fb48941f0\") " pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:19:58.355243 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.355199    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"]
Jul 27 20:19:58.358450 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.358434    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:19:58.360614 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.360594    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-875sf\""
Jul 27 20:19:58.365725 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.365561    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"]
Jul 27 20:19:58.386364 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.386338    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjb7g\" (UniqueName: \"kubernetes.io/projected/28476f03-603f-44f5-8251-841fb48941f0-kube-api-access-kjb7g\") pod \"kuadrant-operator-catalog-vxjl4\" (UID: \"28476f03-603f-44f5-8251-841fb48941f0\") " pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:19:58.394210 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.394181    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjb7g\" (UniqueName: \"kubernetes.io/projected/28476f03-603f-44f5-8251-841fb48941f0-kube-api-access-kjb7g\") pod \"kuadrant-operator-catalog-vxjl4\" (UID: \"28476f03-603f-44f5-8251-841fb48941f0\") " pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:19:58.487453 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.487372    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d52fd6-d55d-4cf1-8574-ffa99edab673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8\" (UID: \"86d52fd6-d55d-4cf1-8574-ffa99edab673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:19:58.510273 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.510250    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:19:58.588260 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.588232    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d52fd6-d55d-4cf1-8574-ffa99edab673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8\" (UID: \"86d52fd6-d55d-4cf1-8574-ffa99edab673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:19:58.588835 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.588816    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d52fd6-d55d-4cf1-8574-ffa99edab673-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8\" (UID: \"86d52fd6-d55d-4cf1-8574-ffa99edab673\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:19:58.642661 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.642638    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxjl4"]
Jul 27 20:19:58.645242 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:19:58.645213    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28476f03_603f_44f5_8251_841fb48941f0.slice/crio-7d7edc8e785e5dee9413f8bbf2bc2b30bab3e01f0507187c76d373237785d8e2 WatchSource:0}: Error finding container 7d7edc8e785e5dee9413f8bbf2bc2b30bab3e01f0507187c76d373237785d8e2: Status 404 returned error can't find the container with id 7d7edc8e785e5dee9413f8bbf2bc2b30bab3e01f0507187c76d373237785d8e2
Jul 27 20:19:58.670994 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.670957    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:19:58.806512 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:58.806491    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"]
Jul 27 20:19:58.808539 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:19:58.808501    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d52fd6_d55d_4cf1_8574_ffa99edab673.slice/crio-ce6efd4913a0a01f3feef16580fdf5c817a5db8a1a476eed0d07a3ea7338a640 WatchSource:0}: Error finding container ce6efd4913a0a01f3feef16580fdf5c817a5db8a1a476eed0d07a3ea7338a640: Status 404 returned error can't find the container with id ce6efd4913a0a01f3feef16580fdf5c817a5db8a1a476eed0d07a3ea7338a640
Jul 27 20:19:59.478824 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:59.478787    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4" event={"ID":"28476f03-603f-44f5-8251-841fb48941f0","Type":"ContainerStarted","Data":"3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e"}
Jul 27 20:19:59.479203 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:59.478845    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4" event={"ID":"28476f03-603f-44f5-8251-841fb48941f0","Type":"ContainerStarted","Data":"7d7edc8e785e5dee9413f8bbf2bc2b30bab3e01f0507187c76d373237785d8e2"}
Jul 27 20:19:59.480465 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:59.480438    2544 generic.go:356] "Generic (PLEG): container finished" podID="86d52fd6-d55d-4cf1-8574-ffa99edab673" containerID="14a0e56cacf3191f169b5a4cb658d9b3b21c3e26aa9bcb4e925150b3c3d00a90" exitCode=0
Jul 27 20:19:59.480587 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:59.480492    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8" event={"ID":"86d52fd6-d55d-4cf1-8574-ffa99edab673","Type":"ContainerDied","Data":"14a0e56cacf3191f169b5a4cb658d9b3b21c3e26aa9bcb4e925150b3c3d00a90"}
Jul 27 20:19:59.480587 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:59.480536    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8" event={"ID":"86d52fd6-d55d-4cf1-8574-ffa99edab673","Type":"ContainerStarted","Data":"ce6efd4913a0a01f3feef16580fdf5c817a5db8a1a476eed0d07a3ea7338a640"}
Jul 27 20:19:59.495349 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:19:59.495308    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4" podStartSLOduration=1.125559422 podStartE2EDuration="1.495294064s" totalImagesPullingTime="369.734642ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:19:59.49359413 +0000 UTC m=+3218.151303662" watchObservedRunningTime="2026-07-27 20:19:59.495294064 +0000 UTC m=+3218.153003616"
Jul 27 20:20:00.627374 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:00.627348    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:20:00.704341 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:00.704310    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d52fd6-d55d-4cf1-8574-ffa99edab673-maas-api-service-ca\") pod \"86d52fd6-d55d-4cf1-8574-ffa99edab673\" (UID: \"86d52fd6-d55d-4cf1-8574-ffa99edab673\") "
Jul 27 20:20:00.704674 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:00.704653    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d52fd6-d55d-4cf1-8574-ffa99edab673-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86d52fd6-d55d-4cf1-8574-ffa99edab673" (UID: "86d52fd6-d55d-4cf1-8574-ffa99edab673"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:00.805617 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:00.805553    2544 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d52fd6-d55d-4cf1-8574-ffa99edab673-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:01.491587 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:01.491557    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"
Jul 27 20:20:01.491776 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:01.491595    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8" event={"ID":"86d52fd6-d55d-4cf1-8574-ffa99edab673","Type":"ContainerDied","Data":"ce6efd4913a0a01f3feef16580fdf5c817a5db8a1a476eed0d07a3ea7338a640"}
Jul 27 20:20:01.491776 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:01.491644    2544 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce6efd4913a0a01f3feef16580fdf5c817a5db8a1a476eed0d07a3ea7338a640"
Jul 27 20:20:08.511145 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:08.511105    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:20:08.511549 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:08.511172    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:20:08.533805 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:08.533710    2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:20:08.945314 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:08.945276    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"]
Jul 27 20:20:08.947178 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:08.947156    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-d3c546-snbs8"]
Jul 27 20:20:09.556949 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:09.556919    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:20:09.592390 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:09.592356    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxjl4"]
Jul 27 20:20:09.840645 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:09.840567    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d52fd6-d55d-4cf1-8574-ffa99edab673" path="/var/lib/kubelet/pods/86d52fd6-d55d-4cf1-8574-ffa99edab673/volumes"
Jul 27 20:20:11.545819 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:11.545784    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4" podUID="28476f03-603f-44f5-8251-841fb48941f0" containerName="registry-server" containerID="cri-o://3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e" gracePeriod=2
Jul 27 20:20:11.813795 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:11.813766    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:20:11.910454 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:11.910417    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjb7g\" (UniqueName: \"kubernetes.io/projected/28476f03-603f-44f5-8251-841fb48941f0-kube-api-access-kjb7g\") pod \"28476f03-603f-44f5-8251-841fb48941f0\" (UID: \"28476f03-603f-44f5-8251-841fb48941f0\") "
Jul 27 20:20:11.912487 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:11.912449    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28476f03-603f-44f5-8251-841fb48941f0-kube-api-access-kjb7g" (OuterVolumeSpecName: "kube-api-access-kjb7g") pod "28476f03-603f-44f5-8251-841fb48941f0" (UID: "28476f03-603f-44f5-8251-841fb48941f0"). InnerVolumeSpecName "kube-api-access-kjb7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:12.011239 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.011204    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjb7g\" (UniqueName: \"kubernetes.io/projected/28476f03-603f-44f5-8251-841fb48941f0-kube-api-access-kjb7g\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:12.552379 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.552340    2544 generic.go:356] "Generic (PLEG): container finished" podID="28476f03-603f-44f5-8251-841fb48941f0" containerID="3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e" exitCode=0
Jul 27 20:20:12.552379 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.552382    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4" event={"ID":"28476f03-603f-44f5-8251-841fb48941f0","Type":"ContainerDied","Data":"3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e"}
Jul 27 20:20:12.552961 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.552404    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4"
Jul 27 20:20:12.552961 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.552458    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vxjl4" event={"ID":"28476f03-603f-44f5-8251-841fb48941f0","Type":"ContainerDied","Data":"7d7edc8e785e5dee9413f8bbf2bc2b30bab3e01f0507187c76d373237785d8e2"}
Jul 27 20:20:12.552961 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.552479    2544 scope.go:117] "RemoveContainer" containerID="3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e"
Jul 27 20:20:12.562491 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.562470    2544 scope.go:117] "RemoveContainer" containerID="3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e"
Jul 27 20:20:12.562859 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:20:12.562832    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e\": container with ID starting with 3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e not found: ID does not exist" containerID="3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e"
Jul 27 20:20:12.562948 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.562871    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e"} err="failed to get container status \"3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e\": rpc error: code = NotFound desc = could not find container \"3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e\": container with ID starting with 3ead18b3ee811376c0fa655afe95a2490ee65d8f6b4321955b9ac584531f690e not found: ID does not exist"
Jul 27 20:20:12.574669 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.574627    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxjl4"]
Jul 27 20:20:12.577315 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:12.577283    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vxjl4"]
Jul 27 20:20:13.840556 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:13.840484    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28476f03-603f-44f5-8251-841fb48941f0" path="/var/lib/kubelet/pods/28476f03-603f-44f5-8251-841fb48941f0/volumes"
Jul 27 20:20:33.364803 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:33.364767    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"]
Jul 27 20:20:33.365290 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:33.365084    2544 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy" containerID="cri-o://e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5" gracePeriod=30
Jul 27 20:20:38.625868 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.625807    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:20:38.692759 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.692711    2544 generic.go:356] "Generic (PLEG): container finished" podID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerID="e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5" exitCode=0
Jul 27 20:20:38.692911 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.692770    2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"
Jul 27 20:20:38.692911 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.692794    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" event={"ID":"edb370f0-a4ef-4708-aa76-2b2037aaad7a","Type":"ContainerDied","Data":"e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5"}
Jul 27 20:20:38.692911 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.692844    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg" event={"ID":"edb370f0-a4ef-4708-aa76-2b2037aaad7a","Type":"ContainerDied","Data":"602800a893bfc5b8cbe3dcf16e137942fe0aa0dc295cf0f6a91ccea05f4b5c10"}
Jul 27 20:20:38.692911 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.692866    2544 scope.go:117] "RemoveContainer" containerID="e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5"
Jul 27 20:20:38.701301 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.701287    2544 scope.go:117] "RemoveContainer" containerID="2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"
Jul 27 20:20:38.708590 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.708573    2544 scope.go:117] "RemoveContainer" containerID="e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5"
Jul 27 20:20:38.708861 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:20:38.708838    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5\": container with ID starting with e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5 not found: ID does not exist" containerID="e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5"
Jul 27 20:20:38.708963 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.708870    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5"} err="failed to get container status \"e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5\": rpc error: code = NotFound desc = could not find container \"e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5\": container with ID starting with e9f30e7b56fd4915b684bdd589023d476bc2a8755a5c5b97e104afa963c50ee5 not found: ID does not exist"
Jul 27 20:20:38.708963 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.708892    2544 scope.go:117] "RemoveContainer" containerID="2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"
Jul 27 20:20:38.709161 ip-10-0-128-237 kubenswrapper[2544]: E0727 20:20:38.709135    2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81\": container with ID starting with 2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81 not found: ID does not exist" containerID="2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"
Jul 27 20:20:38.709199 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.709167    2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81"} err="failed to get container status \"2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81\": rpc error: code = NotFound desc = could not find container \"2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81\": container with ID starting with 2e696365e079d19936cc06d8c2190408ffb4b86e71ec74a415f1352c3c3e5c81 not found: ID does not exist"
Jul 27 20:20:38.743771 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743754    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-token\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.743868 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743797    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-credential-socket\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.743868 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743830    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-podinfo\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.743980 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743885    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-certs\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.743980 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743908    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-envoy\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.743980 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743936    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-socket\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.743980 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.743960    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istiod-ca-cert\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.744182 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744013    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8mh2\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-kube-api-access-t8mh2\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.744182 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744047    2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-data\") pod \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\" (UID: \"edb370f0-a4ef-4708-aa76-2b2037aaad7a\") "
Jul 27 20:20:38.744182 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744053    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:20:38.744337 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744239    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:20:38.744397 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744358    2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.744397 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744370    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:20:38.744508 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.744457    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 20:20:38.745172 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.745151    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:20:38.746306 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.746281    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 20:20:38.746434 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.746418    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:38.746504 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.746466    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-kube-api-access-t8mh2" (OuterVolumeSpecName: "kube-api-access-t8mh2") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "kube-api-access-t8mh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 20:20:38.746504 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.746486    2544 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "edb370f0-a4ef-4708-aa76-2b2037aaad7a" (UID: "edb370f0-a4ef-4708-aa76-2b2037aaad7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 27 20:20:38.845127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845099    2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845122    2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845127 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845131    2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845322 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845139    2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845322 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845147    2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845322 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845156    2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8mh2\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-kube-api-access-t8mh2\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845322 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845164    2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:38.845322 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:38.845172    2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb370f0-a4ef-4708-aa76-2b2037aaad7a-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Jul 27 20:20:39.023678 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.023653    2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"]
Jul 27 20:20:39.028604 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.028576    2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-73ce16-openshift-default-56c85b55b-kdlqg"]
Jul 27 20:20:39.297744 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.297652    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g82mq_440c080f-3966-451b-912b-9d8881cbddb6/manager/0.log"
Jul 27 20:20:39.412596 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.412566    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gwwfj_8cbec7e3-7c5d-4624-a075-07fdf2561442/manager/0.log"
Jul 27 20:20:39.535719 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.535697    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-686ff9dc49-hvg25_3b343eaa-f4c0-4784-976d-4dbbeaec5663/manager/0.log"
Jul 27 20:20:39.652986 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.652963    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-klgvp_28bc3dd5-b6ea-46ef-8142-d202775c9036/server/0.log"
Jul 27 20:20:39.767515 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.767492    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ntf9s_693be74d-32c6-4fa1-ac4b-ce702bfc3910/manager/0.log"
Jul 27 20:20:39.839740 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.839710    2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" path="/var/lib/kubelet/pods/edb370f0-a4ef-4708-aa76-2b2037aaad7a/volumes"
Jul 27 20:20:39.879386 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.879368    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-486qn_b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7/copy-manifests/0.log"
Jul 27 20:20:39.893165 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:39.893145    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-486qn_b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7/manager/0.log"
Jul 27 20:20:40.481944 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:40.481919    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-59d6587f6d-qwh5v_ccf9bee0-6077-4dc7-9b60-16bbaf329e3b/maas-api/0.log"
Jul 27 20:20:40.923075 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:40.923040    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jpgsq_f6f012ef-7b82-4569-b246-d104fb813b07/postgres/0.log"
Jul 27 20:20:42.297203 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:42.297170    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ldrk7_ae5f9549-f879-4175-91e2-8d948532304d/manager/0.log"
Jul 27 20:20:42.628400 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:42.628373    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5k95s_49a9e7ab-4829-44b4-a48e-2d478238c739/registry-server/0.log"
Jul 27 20:20:42.973789 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:42.973702    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-th5td_436f90f9-8233-4074-b687-2e6bf5f3861a/manager/0.log"
Jul 27 20:20:43.532143 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:43.532110    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-mgjtv_f61a9a12-e963-4935-8e03-b67c0b35f921/kube-auth-proxy/0.log"
Jul 27 20:20:43.743583 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:43.743548    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/1.log"
Jul 27 20:20:43.865530 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:43.865500    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ct46q_3faa46e7-6a90-42b3-89c0-253e7ed878d6/payload-pre-processing/0.log"
Jul 27 20:20:43.974619 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:43.974592    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gbhk5_602f0fdc-6d53-45ad-a048-112e95eb5f7b/payload-processing/0.log"
Jul 27 20:20:44.085387 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:44.085343    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-645bb56f9f-b8xdb_2e0d2ae6-7377-4a43-8430-3b65150f65af/router/0.log"
Jul 27 20:20:44.413229 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:44.413205    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2jxgm_517465b2-220d-4d59-8642-ba177f09a704/main/0.log"
Jul 27 20:20:44.528198 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:44.528169    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6z7zc_b8638429-e9da-4447-bac3-fd6393a74fa9/main/0.log"
Jul 27 20:20:44.962385 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:44.962341    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-hjfcz_4098324b-2360-401f-9684-7ba7ab404f24/main/0.log"
Jul 27 20:20:52.652298 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:52.652266    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jks2p_efd9f735-7bea-4f49-8d53-b62a27a10da3/global-pull-secret-syncer/0.log"
Jul 27 20:20:52.768495 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:52.768471    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pw577_87c5dfeb-d7d2-41df-9415-2de9e282413b/konnectivity-agent/0.log"
Jul 27 20:20:52.817657 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:52.817624    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-237.ec2.internal_3b923290a23099a90256d2af76020e79/haproxy/0.log"
Jul 27 20:20:57.446620 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:57.446593    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ldrk7_ae5f9549-f879-4175-91e2-8d948532304d/manager/0.log"
Jul 27 20:20:57.581981 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:57.581942    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5k95s_49a9e7ab-4829-44b4-a48e-2d478238c739/registry-server/0.log"
Jul 27 20:20:57.861280 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:57.861253    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-th5td_436f90f9-8233-4074-b687-2e6bf5f3861a/manager/0.log"
Jul 27 20:20:59.676953 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:59.676896    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-6nhm9_4fceebef-ef4a-4f7d-af19-a773ba51379f/cluster-monitoring-operator/0.log"
Jul 27 20:20:59.833550 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:59.833520    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46qdz_54baa635-9a8c-43a0-bd9e-6c1cbb63b156/node-exporter/0.log"
Jul 27 20:20:59.854121 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:59.854084    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46qdz_54baa635-9a8c-43a0-bd9e-6c1cbb63b156/kube-rbac-proxy/0.log"
Jul 27 20:20:59.875191 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:20:59.875173    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-46qdz_54baa635-9a8c-43a0-bd9e-6c1cbb63b156/init-textfile/0.log"
Jul 27 20:21:02.201951 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:02.201923    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-4hhz6_9fd028b2-a23b-416a-92bc-2f7e15ae1ae7/networking-console-plugin/0.log"
Jul 27 20:21:02.882125 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:02.882098    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/1.log"
Jul 27 20:21:02.886685 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:02.886662    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-dwtww_bfbcdbee-4f5a-49ca-bb43-157da996dad9/console-operator/2.log"
Jul 27 20:21:03.415986 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.415956    2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"]
Jul 27 20:21:03.416439 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416412    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy"
Jul 27 20:21:03.416439 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416439    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy"
Jul 27 20:21:03.416569 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416455    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy"
Jul 27 20:21:03.416569 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416460    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy"
Jul 27 20:21:03.416569 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416472    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d52fd6-d55d-4cf1-8574-ffa99edab673" containerName="revoke-keys"
Jul 27 20:21:03.416569 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416481    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d52fd6-d55d-4cf1-8574-ffa99edab673" containerName="revoke-keys"
Jul 27 20:21:03.416569 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416491    2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28476f03-603f-44f5-8251-841fb48941f0" containerName="registry-server"
Jul 27 20:21:03.416569 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416496    2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="28476f03-603f-44f5-8251-841fb48941f0" containerName="registry-server"
Jul 27 20:21:03.416801 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416583    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="28476f03-603f-44f5-8251-841fb48941f0" containerName="registry-server"
Jul 27 20:21:03.416801 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416592    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy"
Jul 27 20:21:03.416801 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416599    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb370f0-a4ef-4708-aa76-2b2037aaad7a" containerName="istio-proxy"
Jul 27 20:21:03.416801 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.416607    2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d52fd6-d55d-4cf1-8574-ffa99edab673" containerName="revoke-keys"
Jul 27 20:21:03.419596 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.419573    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.421938 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.421918    2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gzr5q\"/\"default-dockercfg-x6f4j\""
Jul 27 20:21:03.422218 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.422201    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzr5q\"/\"openshift-service-ca.crt\""
Jul 27 20:21:03.423054 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.423035    2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gzr5q\"/\"kube-root-ca.crt\""
Jul 27 20:21:03.429460 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.429437    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"]
Jul 27 20:21:03.550247 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.550220    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-lib-modules\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.550357 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.550267    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-sys\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.550357 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.550339    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-podres\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.550452 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.550427    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlb6g\" (UniqueName: \"kubernetes.io/projected/5c897623-da34-4bbc-88aa-7733046562b2-kube-api-access-nlb6g\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.550493 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.550448    2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-proc\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651317 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651288    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlb6g\" (UniqueName: \"kubernetes.io/projected/5c897623-da34-4bbc-88aa-7733046562b2-kube-api-access-nlb6g\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651429 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651328    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-proc\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651429 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651408    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-lib-modules\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651541 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651455    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-sys\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651541 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651480    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-proc\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651541 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651484    2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-podres\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651674 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651576    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-sys\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651674 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651600    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-podres\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.651674 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.651609    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c897623-da34-4bbc-88aa-7733046562b2-lib-modules\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.659413 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.659391    2544 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlb6g\" (UniqueName: \"kubernetes.io/projected/5c897623-da34-4bbc-88aa-7733046562b2-kube-api-access-nlb6g\") pod \"perf-node-gather-daemonset-qlhrc\" (UID: \"5c897623-da34-4bbc-88aa-7733046562b2\") " pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.730870 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.730815    2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:03.861369 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:03.861347    2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"]
Jul 27 20:21:03.863405 ip-10-0-128-237 kubenswrapper[2544]: W0727 20:21:03.863369    2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5c897623_da34_4bbc_88aa_7733046562b2.slice/crio-7d5684c0ec6e2f80cd493cf9e2be881170e93bb20374015c6316e7becefb8373 WatchSource:0}: Error finding container 7d5684c0ec6e2f80cd493cf9e2be881170e93bb20374015c6316e7becefb8373: Status 404 returned error can't find the container with id 7d5684c0ec6e2f80cd493cf9e2be881170e93bb20374015c6316e7becefb8373
Jul 27 20:21:04.193139 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:04.193115    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-92c5x_ee3cc093-676d-4825-b54b-0a95e073bde6/volume-data-source-validator/0.log"
Jul 27 20:21:04.840189 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:04.840142    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc" event={"ID":"5c897623-da34-4bbc-88aa-7733046562b2","Type":"ContainerStarted","Data":"b94c70354560f86d8f30c3cb34eb10bb780719e78f532103e4830ef7ffd9e298"}
Jul 27 20:21:04.840624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:04.840210    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:04.840624 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:04.840232    2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc" event={"ID":"5c897623-da34-4bbc-88aa-7733046562b2","Type":"ContainerStarted","Data":"7d5684c0ec6e2f80cd493cf9e2be881170e93bb20374015c6316e7becefb8373"}
Jul 27 20:21:04.856600 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:04.856558    2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc" podStartSLOduration=1.856544239 podStartE2EDuration="1.856544239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:21:04.855081164 +0000 UTC m=+3283.512790692" watchObservedRunningTime="2026-07-27 20:21:04.856544239 +0000 UTC m=+3283.514253769"
Jul 27 20:21:05.290645 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:05.290622    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zqrl2_2f225d20-1fb1-4cd0-bf76-31d0b51f791f/dns/0.log"
Jul 27 20:21:05.311050 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:05.311025    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zqrl2_2f225d20-1fb1-4cd0-bf76-31d0b51f791f/kube-rbac-proxy/0.log"
Jul 27 20:21:05.333552 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:05.333527    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-757g8_088f9643-e34a-468f-b1be-432eaa0d2346/dns-node-resolver/0.log"
Jul 27 20:21:06.009113 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:06.009088    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xg9lr_78ac947a-ae79-4d31-a502-ec3efb1aafd6/node-ca/0.log"
Jul 27 20:21:07.073172 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:07.073148    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-mgjtv_f61a9a12-e963-4935-8e03-b67c0b35f921/kube-auth-proxy/0.log"
Jul 27 20:21:07.125236 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:07.125212    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/0.log"
Jul 27 20:21:07.154166 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:07.154143    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/1.log"
Jul 27 20:21:07.183197 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:07.183178    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ct46q_3faa46e7-6a90-42b3-89c0-253e7ed878d6/payload-pre-processing/0.log"
Jul 27 20:21:07.211309 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:07.211288    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gbhk5_602f0fdc-6d53-45ad-a048-112e95eb5f7b/payload-processing/0.log"
Jul 27 20:21:07.231870 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:07.231809    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-645bb56f9f-b8xdb_2e0d2ae6-7377-4a43-8430-3b65150f65af/router/0.log"
Jul 27 20:21:08.069094 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:08.069069    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nxgrc_07f8c00f-e1ad-4805-8ec3-3465493eddca/serve-healthcheck-canary/0.log"
Jul 27 20:21:08.529756 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:08.529698    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-62tqr_c6b55b08-2fc0-4416-bba2-bd4f0d5f6308/insights-operator/0.log"
Jul 27 20:21:08.532141 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:08.532120    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-62tqr_c6b55b08-2fc0-4416-bba2-bd4f0d5f6308/insights-operator/1.log"
Jul 27 20:21:08.617246 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:08.617228    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-92r4h_0209970c-c0d2-4c96-87ef-9e27ffa077b6/kube-rbac-proxy/0.log"
Jul 27 20:21:08.637618 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:08.637600    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-92r4h_0209970c-c0d2-4c96-87ef-9e27ffa077b6/exporter/0.log"
Jul 27 20:21:08.660363 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:08.660349    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-92r4h_0209970c-c0d2-4c96-87ef-9e27ffa077b6/extractor/0.log"
Jul 27 20:21:10.825518 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:10.825486    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-g82mq_440c080f-3966-451b-912b-9d8881cbddb6/manager/0.log"
Jul 27 20:21:10.855226 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:10.855193    2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gzr5q/perf-node-gather-daemonset-qlhrc"
Jul 27 20:21:11.041137 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:11.041111    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-gwwfj_8cbec7e3-7c5d-4624-a075-07fdf2561442/manager/0.log"
Jul 27 20:21:11.165970 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:11.165947    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-686ff9dc49-hvg25_3b343eaa-f4c0-4784-976d-4dbbeaec5663/manager/0.log"
Jul 27 20:21:11.185491 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:11.185473    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-klgvp_28bc3dd5-b6ea-46ef-8142-d202775c9036/server/0.log"
Jul 27 20:21:11.261284 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:11.261262    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ntf9s_693be74d-32c6-4fa1-ac4b-ce702bfc3910/manager/0.log"
Jul 27 20:21:11.343820 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:11.343797    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-486qn_b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7/manager/0.log"
Jul 27 20:21:11.360447 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:11.360422    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c9f4fc9b4-486qn_b5ad9ecc-5d9f-48e7-a09f-0ecb66b4a0d7/copy-manifests/0.log"
Jul 27 20:21:13.752023 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:13.751996    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-mgjtv_f61a9a12-e963-4935-8e03-b67c0b35f921/kube-auth-proxy/0.log"
Jul 27 20:21:13.785450 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:13.785428    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/0.log"
Jul 27 20:21:13.812551 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:13.812530    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/1.log"
Jul 27 20:21:13.834524 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:13.834475    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ct46q_3faa46e7-6a90-42b3-89c0-253e7ed878d6/payload-pre-processing/0.log"
Jul 27 20:21:13.854459 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:13.854439    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gbhk5_602f0fdc-6d53-45ad-a048-112e95eb5f7b/payload-processing/0.log"
Jul 27 20:21:13.867975 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:13.867957    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-645bb56f9f-b8xdb_2e0d2ae6-7377-4a43-8430-3b65150f65af/router/0.log"
Jul 27 20:21:15.488915 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:15.488889    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b7cdcf968-mgjtv_f61a9a12-e963-4935-8e03-b67c0b35f921/kube-auth-proxy/0.log"
Jul 27 20:21:15.530267 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:15.530226    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/0.log"
Jul 27 20:21:15.554046 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:15.554025    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-gxwfr_b8a643f3-33b1-4b82-8021-0f632ef6a3a2/istio-proxy/1.log"
Jul 27 20:21:15.579937 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:15.579916    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-ct46q_3faa46e7-6a90-42b3-89c0-253e7ed878d6/payload-pre-processing/0.log"
Jul 27 20:21:15.603796 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:15.603780    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gbhk5_602f0fdc-6d53-45ad-a048-112e95eb5f7b/payload-processing/0.log"
Jul 27 20:21:15.621966 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:15.621938    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-645bb56f9f-b8xdb_2e0d2ae6-7377-4a43-8430-3b65150f65af/router/0.log"
Jul 27 20:21:18.572216 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:18.572181    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5_0af76269-e664-48e9-addb-350ebc791abd/kube-storage-version-migrator-operator/1.log"
Jul 27 20:21:18.573118 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:18.573095    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-rmhs5_0af76269-e664-48e9-addb-350ebc791abd/kube-storage-version-migrator-operator/0.log"
Jul 27 20:21:20.008689 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.008658    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/kube-multus-additional-cni-plugins/0.log"
Jul 27 20:21:20.029807 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.029784    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/egress-router-binary-copy/0.log"
Jul 27 20:21:20.051049 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.051031    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/cni-plugins/0.log"
Jul 27 20:21:20.072387 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.072364    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/bond-cni-plugin/0.log"
Jul 27 20:21:20.095816 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.095801    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/routeoverride-cni/0.log"
Jul 27 20:21:20.116413 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.116396    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/whereabouts-cni-bincopy/0.log"
Jul 27 20:21:20.136154 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.136136    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xk5wh_e851f244-643c-4512-97cc-ca6e43bbbce6/whereabouts-cni/0.log"
Jul 27 20:21:20.170842 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.170821    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qqs98_5fb7d020-8ad1-4b43-b7a5-1099d16ebd9c/kube-multus/0.log"
Jul 27 20:21:20.190200 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.190183    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7d4b9_f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01/network-metrics-daemon/0.log"
Jul 27 20:21:20.212000 ip-10-0-128-237 kubenswrapper[2544]: I0727 20:21:20.211983    2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7d4b9_f5a7a022-6cc9-4aca-8c4d-8e18d93d1a01/kube-rbac-proxy/0.log"