Jul 07 17:51:56.147725 ip-10-0-134-243 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 17:51:56.147740 ip-10-0-134-243 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 17:51:56.147750 ip-10-0-134-243 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 17:51:56.148067 ip-10-0-134-243 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 17:52:06.214793 ip-10-0-134-243 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 17:52:06.214811 ip-10-0-134-243 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot acefbc034a2f40bd83a638d655a3138b --
Jul 07 17:53:57.996238 ip-10-0-134-243 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 17:53:58.369872 ip-10-0-134-243 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 17:53:58.369872 ip-10-0-134-243 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 17:53:58.369872 ip-10-0-134-243 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 17:53:58.369872 ip-10-0-134-243 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 17:53:58.369872 ip-10-0-134-243 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 17:53:58.370515 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.369876    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 17:53:58.374721 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374702    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:53:58.374721 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374720    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:53:58.374721 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374724    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374728    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374731    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374736    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374739    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374742    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374744    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374747    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374749    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374752    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374754    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374757    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374759    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374762    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374764    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374767    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374770    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374773    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374775    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374779    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:53:58.374816 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374781    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374784    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374786    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374789    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374791    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374794    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374796    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374799    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374801    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374804    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374807    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374809    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374812    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374814    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374819    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374822    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374825    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374828    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374831    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374833    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:53:58.375311 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374836    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374839    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374842    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374844    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374847    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374849    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374852    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374856    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374861    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374864    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374867    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374869    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374872    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374875    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374877    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374880    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374882    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374885    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374888    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374890    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:53:58.376066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374893    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374895    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374898    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374918    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374923    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374927    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374933    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374939    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374943    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374946    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374949    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374951    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374954    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374957    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374959    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374964    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374967    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374970    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374973    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374975    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:53:58.376750 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374978    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374980    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374983    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374985    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374988    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374990    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374992    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374995    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.374998    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375001    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375005    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375008    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375010    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375013    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375015    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375018    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:53:58.377264 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.375020    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377337    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377352    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377355    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377359    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377362    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377366    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377368    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377371    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377374    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377377    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377381    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377385    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377388    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377391    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377394    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377396    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377399    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377402    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:53:58.377646 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377404    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377407    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377409    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377412    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377414    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377417    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377420    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377422    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377425    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377427    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377430    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377433    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377435    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377439    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377441    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377445    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377448    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377450    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377453    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:53:58.378126 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377456    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377459    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377461    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377464    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377466    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377469    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377471    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377474    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377476    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377478    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377481    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377484    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377487    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377491    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377494    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377496    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377499    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377501    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377504    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377506    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:53:58.378591 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377509    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377511    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377514    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377517    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377522    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377526    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377530    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377532    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377535    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377538    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377540    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377543    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377546    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377549    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377551    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377554    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377557    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377560    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377562    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377565    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:53:58.379088 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377567    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377570    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377572    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377575    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377577    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377581    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377585    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377587    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377590    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377593    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377595    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377598    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377600    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377603    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377605    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377608    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377612    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377616    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377619    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377622    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:53:58.379608 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377624    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.377627    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378139    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378145    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378148    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378152    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378154    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378157    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378160    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378162    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378165    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378168    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378170    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378173    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378176    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378179    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378181    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378185    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378189    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378192    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:53:58.380101 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378195    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378197    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378200    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378202    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378205    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378208    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378210    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378213    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378216    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378218    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378221    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378224    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378226    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378229    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378232    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378234    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378237    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378239    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378242    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378244    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:53:58.380581 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378247    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378249    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378252    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378254    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378257    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378259    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378262    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378264    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378267    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378270    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378272    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378275    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378278    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378280    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378283    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378285    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378288    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378291    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378294    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378297    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:53:58.381113 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378300    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378302    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378304    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378307    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378309    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378314    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378317    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378320    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378323    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378326    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378329    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378331    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378334    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378337    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378341    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378344    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378348    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378351    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378353    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:53:58.381595 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378357    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378361    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378364    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378366    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378369    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378373    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378375    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378379    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378382    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378385    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378388    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378391    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378394    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378396    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378399    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378401    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378404    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378406    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378409    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:53:58.382066 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378412    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378414    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.378417    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378501    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378511    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378520    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378536    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378542    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378546    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378551    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378555    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378559    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378562    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378566    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378570    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378573    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378576    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378578    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378581    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378585    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378588    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378593    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378598    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 17:53:58.382560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378602    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378605    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378608    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378616    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378619    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378622    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378626    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378629    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378632    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378635    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378638    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378640    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378645    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378648    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378651    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378654    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378657    2580 flags.go:64] FLAG: --enable-server="true"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378659    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378664    2580 flags.go:64] FLAG: --event-burst="100"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378667    2580 flags.go:64] FLAG: --event-qps="50"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378670    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378673    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378676    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378680    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 17:53:58.383200 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378683    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378686    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378688    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378691    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378694    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378698    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378701    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378705    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378708    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378710    2580 flags.go:64] FLAG: --feature-gates=""
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378714    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378717    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378720    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378723    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378726    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378729    2580 flags.go:64] FLAG: --help="false"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378732    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378735    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378738    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378741    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378744    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378747    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378750    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 17:53:58.383770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378753    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378756    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378759    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378761    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378765    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378767    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378770    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378773    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378776    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378779    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378781    2580 flags.go:64] FLAG: --lock-file=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378784    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378787    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378790    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378795    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378799    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378803    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378806    2580 flags.go:64] FLAG: --logging-format="text"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378809    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378812    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378815    2580 flags.go:64] FLAG: --manifest-url=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378818    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378823    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378825    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378830    2580 flags.go:64] FLAG: --max-pods="110"
Jul 07 17:53:58.384359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378833    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378836    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378838    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378842    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378844    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378848    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378850    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378858    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378861    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378865    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378867    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378870    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378876    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378878    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378881    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378884    2580 flags.go:64] FLAG: --port="10250"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378887    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378890    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e1b049772fdff226"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378893    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378896    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378899    2580 flags.go:64] FLAG: --register-node="true"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378918    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378922    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378926    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 07 17:53:58.384980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378930    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378932    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378936    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378939    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378943    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378945    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378948    2580 flags.go:64] FLAG: --runonce="false"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378951    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378954    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378957    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378960    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378962    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378965    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378968    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378971    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378974    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378977    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378980    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378982    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378985    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378988    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378991    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378996    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.378999    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379002    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 17:53:58.385565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379006    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379009    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379012    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379014    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379017    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379020    2580 flags.go:64] FLAG: --v="2"
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379024    2580 flags.go:64] FLAG: --version="false"
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379030    2580 flags.go:64] FLAG: --vmodule=""
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379034    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.379038    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379167    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379172    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379175    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379178    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379181    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379183    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379186    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379188    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379191    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379193    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379196    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379199    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:53:58.386217 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379201    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379204    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379206    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379209    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379212    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379216    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379219    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379222    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379226    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379229    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379232    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379234    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379236    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379239    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379242    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379244    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379247    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379250    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379253    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:53:58.386758 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379256    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379258    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379261    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379263    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379266    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379268    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379271    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379273    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379275    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379279    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379282    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379285    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379287    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379290    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379292    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379295    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379298    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379300    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379303    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:53:58.387242 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379306    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379308    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379310    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379313    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379315    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379318    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379320    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379323    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379326    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379328    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379331    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379333    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379335    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379339    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379342    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379344    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379347    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379350    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379352    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379355    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:53:58.387714 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379358    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379360    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379363    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379365    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379368    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379378    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379384    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379387    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379390    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379392    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379395    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379397    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379400    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379403    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379406    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379408    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379410    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379413    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379415    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379418    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:53:58.388288 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379420    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379423    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379425    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379428    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379430    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379433    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379437    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379439    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.379442    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.381354    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.388374    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.388391    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.388410    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.388416    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 17:53:58.388778 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388473    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388478    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388482    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388485    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388488    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388491    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388494    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388497    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388499    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388502    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388505    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388507    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388510    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388514    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388518    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388521    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388524    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388526    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388529    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388532    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:53:58.389143 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388535    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388537    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388540    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388542    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388546    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388549    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388552    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388554    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388557    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388561    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388563    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388566    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388569    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388571    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388574    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388577    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388580    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388582    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388585    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388587    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:53:58.389616 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388589    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388592    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388594    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388597    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388600    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388602    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388606    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388609    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388612    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388615    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388617    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388621    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388625    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388628    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388631    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388633    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388636    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388639    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388642    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:53:58.390120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388644    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388647    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388650    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388653    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388655    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388658    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388660    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388663    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388665    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388668    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388671    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388673    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388675    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388679    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388681    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388684    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388686    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388689    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388691    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388693    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:53:58.390566 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388696    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388698    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388701    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388703    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388706    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388708    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388711    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388713    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388716    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388718    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388721    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388724    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388726    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388729    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388732    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388735    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388737    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388740    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388743    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:53:58.391067 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388745    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.388750    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388865    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388869    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388872    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388875    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388877    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388880    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388883    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388885    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388888    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388891    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388893    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388896    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:53:58.391551 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388898    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388917    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388921    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388924    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388927    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388929    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388932    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388935    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388938    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388940    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388942    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388945    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388948    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388951    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388954    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388956    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388959    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388961    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388964    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388966    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:53:58.391892 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388969    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388971    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388974    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388976    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388979    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388981    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388984    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388986    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388989    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388991    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388994    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388996    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.388999    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389001    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389004    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389006    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389008    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389011    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389013    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:53:58.392456 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389015    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389018    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389020    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389023    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389026    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389030    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389032    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389035    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389037    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389039    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389042    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389044    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389047    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389049    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389052    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389054    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389057    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389059    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389062    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389064    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:53:58.392933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389067    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389070    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389072    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389075    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389077    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389079    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389082    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389085    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389088    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389091    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389094    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389096    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389098    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389101    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389103    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389107    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389110    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389114    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389117    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:53:58.393408 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389119    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389122    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389124    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389127    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389129    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389132    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389134    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389137    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:58.389139    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.389144    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.389265    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.391148    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.393328    2580 server.go:1019] "Starting client certificate rotation"
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.393437    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 17:53:58.393873 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.393471    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 17:53:58.396438 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.396421    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 17:53:58.400080 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.400055    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 17:53:58.411045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.411020    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 07 17:53:58.412851 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.412837    2580 log.go:25] "Validated CRI v1 image API"
Jul 07 17:53:58.413607 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.413592    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 17:53:58.415589 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.415568    2580 fs.go:135] Filesystem UUIDs: map[02238d08-422a-41be-ae67-61742f4ae54e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8789e044-9dc5-4b2d-9ddb-bc2f1360c78e:/dev/nvme0n1p3]
Jul 07 17:53:58.415639 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.415589    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 17:53:58.421291 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.421269    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 17:53:58.421489 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.421362    2580 manager.go:216] Machine: {Timestamp:2026-07-07 17:53:58.419262725 +0000 UTC m=+0.309623440 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098176 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23f1d3a21f935083f980d75608efa0 SystemUUID:ec23f1d3-a21f-9350-83f9-80d75608efa0 BootID:acefbc03-4a2f-40bd-83a6-38d655a3138b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cf:7d:46:12:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cf:7d:46:12:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:94:ce:25:16:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 17:53:58.421489 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.421488    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 17:53:58.421593 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.421569    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 17:53:58.425435 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.425406    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 17:53:58.425598 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.425439    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-243.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 17:53:58.426190 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.426179    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 17:53:58.426221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.426193    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 17:53:58.426221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.426200    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 17:53:58.426845 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.426835    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 17:53:58.428497 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.428486    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 17:53:58.431557 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431546    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 17:53:58.431679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431669    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 17:53:58.431712 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431702    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 17:53:58.431740 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431713    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 17:53:58.431740 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431728    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 17:53:58.431740 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431737    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 07 17:53:58.431842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.431747    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 17:53:58.432121 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.432104    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 17:53:58.432177 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.432139    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 17:53:58.434300 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.434269    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 17:53:58.435977 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.435964    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 17:53:58.436019 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.435996    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 17:53:58.437031 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437017    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 17:53:58.437066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437036    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 17:53:58.437066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437042    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 17:53:58.437066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437047    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 17:53:58.437066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437055    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 17:53:58.437066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437061    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 17:53:58.437221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437069    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 17:53:58.437221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437074    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 17:53:58.437221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437081    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 17:53:58.437221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437086    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 17:53:58.437221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437094    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 17:53:58.437221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437099    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 17:53:58.437745 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437733    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 17:53:58.437745 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.437744    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 17:53:58.440985 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.440969    2580 server.go:1268] "Started kubelet"
Jul 07 17:53:58.441102 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.441074    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 17:53:58.441157 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.441098    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 17:53:58.441211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.441159    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 17:53:58.441616 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.441587    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-243.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 17:53:58.441728 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.441695    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-243.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 17:53:58.441819 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.441793    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 17:53:58.442045 ip-10-0-134-243 systemd[1]: Started Kubernetes Kubelet.
Jul 07 17:53:58.443443 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.443049    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 17:53:58.443651 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.443634    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 17:53:58.446874 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.446845    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 17:53:58.450357 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.449430    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-243.ec2.internal.18c013735110baab  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-243.ec2.internal,UID:ip-10-0-134-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-243.ec2.internal,},FirstTimestamp:2026-07-07 17:53:58.440934059 +0000 UTC m=+0.331294775,LastTimestamp:2026-07-07 17:53:58.440934059 +0000 UTC m=+0.331294775,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-243.ec2.internal,}"
Jul 07 17:53:58.450536 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.450522    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 17:53:58.450612 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.450596    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 17:53:58.450671 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.450653    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:58.451027 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451012    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 17:53:58.451118 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451036    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 17:53:58.451118 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451052    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 17:53:58.451222 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451187    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 17:53:58.451222 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451194    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 17:53:58.451683 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451668    2580 factory.go:55] Registering systemd factory
Jul 07 17:53:58.451748 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451686    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 07 17:53:58.451953 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451937    2580 factory.go:153] Registering CRI-O factory
Jul 07 17:53:58.452026 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451955    2580 factory.go:223] Registration of the crio container factory successfully
Jul 07 17:53:58.452026 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.451995    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 17:53:58.452026 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.452019    2580 factory.go:103] Registering Raw factory
Jul 07 17:53:58.452223 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.452031    2580 manager.go:1196] Started watching for new ooms in manager
Jul 07 17:53:58.452489 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.452421    2580 manager.go:349] Starting recovery of all containers
Jul 07 17:53:58.457798 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.457764    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-243.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 17:53:58.457893 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.457871    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 17:53:58.459962 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.459896    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 17:53:58.463195 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.463177    2580 manager.go:354] Recovery completed
Jul 07 17:53:58.467854 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.467838    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.469754 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.469733    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.469822 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.469772    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.469822 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.469784    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.470308 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.470295    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 17:53:58.470308 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.470307    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 17:53:58.470421 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.470323    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 17:53:58.470421 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.470306    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.471093 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.471078    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.471151 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.471107    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.471151 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.471120    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.471500 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.471489    2580 policy_none.go:49] "None policy: Start"
Jul 07 17:53:58.471549 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.471505    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 17:53:58.471549 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.471516    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 17:53:58.472641 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.472625    2580 policy_none.go:47] "Start"
Jul 07 17:53:58.472722 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.472553    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-243.ec2.internal.18c0137352c88196  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-243.ec2.internal,UID:ip-10-0-134-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-243.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-243.ec2.internal,},FirstTimestamp:2026-07-07 17:53:58.469755286 +0000 UTC m=+0.360116009,LastTimestamp:2026-07-07 17:53:58.469755286 +0000 UTC m=+0.360116009,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-243.ec2.internal,}"
Jul 07 17:53:58.484479 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.484411    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-243.ec2.internal.18c0137352c8d7d2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-243.ec2.internal,UID:ip-10-0-134-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-243.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-243.ec2.internal,},FirstTimestamp:2026-07-07 17:53:58.469777362 +0000 UTC m=+0.360138077,LastTimestamp:2026-07-07 17:53:58.469777362 +0000 UTC m=+0.360138077,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-243.ec2.internal,}"
Jul 07 17:53:58.489645 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.489625    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8rbnx"
Jul 07 17:53:58.494989 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.494921    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-243.ec2.internal.18c0137352c90053  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-243.ec2.internal,UID:ip-10-0-134-243.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-243.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-243.ec2.internal,},FirstTimestamp:2026-07-07 17:53:58.469787731 +0000 UTC m=+0.360148446,LastTimestamp:2026-07-07 17:53:58.469787731 +0000 UTC m=+0.360148446,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-243.ec2.internal,}"
Jul 07 17:53:58.498192 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.498170    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8rbnx"
Jul 07 17:53:58.507540 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.507519    2580 manager.go:337] "Starting Device Plugin manager"
Jul 07 17:53:58.507643 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.507557    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 17:53:58.507643 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.507569    2580 server.go:87] "Starting device plugin registration server"
Jul 07 17:53:58.507827 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.507813    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 17:53:58.507879 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.507829    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 17:53:58.507965 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.507934    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 17:53:58.508037 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.508024    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 17:53:58.508085 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.508037    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 17:53:58.508691 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.508671    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 17:53:58.508765 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.508723    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:58.588280 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.588241    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 17:53:58.588280 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.588287    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 17:53:58.589409 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.588311    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 17:53:58.589409 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.588342    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 17:53:58.590843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.590815    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:53:58.607970 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.607938    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.609136 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.609116    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.609252 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.609148    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.609252 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.609159    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.609252 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.609181    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.616172 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.616153    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.616223 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.616181    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-243.ec2.internal\": node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:58.648141 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.648066    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:58.688758 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.688727    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"]
Jul 07 17:53:58.688883 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.688797    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.689724 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.689710    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.689788 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.689743    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.689788 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.689753    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.691303 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.691290    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.691450 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.691423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.691493 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.691459    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.692021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.691997    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.692021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.692006    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.692021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.692022    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.692021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.692026    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.692227 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.692035    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.692227 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.692036    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.693385 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.693367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.693447 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.693392    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:58.694060 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.694044    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:58.694133 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.694067    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:58.694133 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.694077    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:58.714551 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.714531    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-243.ec2.internal\" not found" node="ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.719038 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.719022    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-243.ec2.internal\" not found" node="ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.748824 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.748799    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:58.754266 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.754242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0953349d49bbfb3ea71d025821a8f5d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-243.ec2.internal\" (UID: \"d0953349d49bbfb3ea71d025821a8f5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.754359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.754278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/afa4c1649d9a92df7d64f13fdc5951de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal\" (UID: \"afa4c1649d9a92df7d64f13fdc5951de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.754359 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.754304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afa4c1649d9a92df7d64f13fdc5951de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal\" (UID: \"afa4c1649d9a92df7d64f13fdc5951de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.849733 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.849696    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:58.855082 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.855059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/afa4c1649d9a92df7d64f13fdc5951de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal\" (UID: \"afa4c1649d9a92df7d64f13fdc5951de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.855175 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.855094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afa4c1649d9a92df7d64f13fdc5951de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal\" (UID: \"afa4c1649d9a92df7d64f13fdc5951de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.855175 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.855151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0953349d49bbfb3ea71d025821a8f5d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-243.ec2.internal\" (UID: \"d0953349d49bbfb3ea71d025821a8f5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.855268 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.855171    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/afa4c1649d9a92df7d64f13fdc5951de-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal\" (UID: \"afa4c1649d9a92df7d64f13fdc5951de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.855268 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.855214    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0953349d49bbfb3ea71d025821a8f5d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-243.ec2.internal\" (UID: \"d0953349d49bbfb3ea71d025821a8f5d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.855268 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:58.855238    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/afa4c1649d9a92df7d64f13fdc5951de-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal\" (UID: \"afa4c1649d9a92df7d64f13fdc5951de\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:58.950579 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:58.950497    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.017946 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.017891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:59.021859 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.021713    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:59.050773 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.050728    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.151248 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.151207    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.251669 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.251597    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.352695 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.352646    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.391728 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.391684    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 17:53:59.392375 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.391891    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 17:53:59.453577 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.453541    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.501048 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.500970    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 17:48:58 +0000 UTC" deadline="2028-01-27 16:24:07.114056441 +0000 UTC"
Jul 07 17:53:59.501048 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.501050    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13654h30m7.613016905s"
Jul 07 17:53:59.506190 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.506122    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:53:59.509645 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.509606    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:53:59.509645 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.509647    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:53:59.509831 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.509662    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:53:59.530980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.530947    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:53:59.554070 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.554022    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.557825 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:59.557788    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0953349d49bbfb3ea71d025821a8f5d.slice/crio-3eda972b2cb85b1f458bff05b7047cfa0c2ceda6777300d11b4b66fc6f57c6d0 WatchSource:0}: Error finding container 3eda972b2cb85b1f458bff05b7047cfa0c2ceda6777300d11b4b66fc6f57c6d0: Status 404 returned error can't find the container with id 3eda972b2cb85b1f458bff05b7047cfa0c2ceda6777300d11b4b66fc6f57c6d0
Jul 07 17:53:59.563259 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.563239    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 17:53:59.567623 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:53:59.567595    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa4c1649d9a92df7d64f13fdc5951de.slice/crio-9147377f04b4f6b2a8b2a7ecac2dff4472e4a37c3156ead603f8418923e16da2 WatchSource:0}: Error finding container 9147377f04b4f6b2a8b2a7ecac2dff4472e4a37c3156ead603f8418923e16da2: Status 404 returned error can't find the container with id 9147377f04b4f6b2a8b2a7ecac2dff4472e4a37c3156ead603f8418923e16da2
Jul 07 17:53:59.592612 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.592548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal" event={"ID":"afa4c1649d9a92df7d64f13fdc5951de","Type":"ContainerStarted","Data":"9147377f04b4f6b2a8b2a7ecac2dff4472e4a37c3156ead603f8418923e16da2"}
Jul 07 17:53:59.593724 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.593699    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal" event={"ID":"d0953349d49bbfb3ea71d025821a8f5d","Type":"ContainerStarted","Data":"3eda972b2cb85b1f458bff05b7047cfa0c2ceda6777300d11b4b66fc6f57c6d0"}
Jul 07 17:53:59.648519 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.648488    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:53:59.654140 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.654106    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.754388 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:53:59.754342    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-243.ec2.internal\" not found"
Jul 07 17:53:59.828634 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.828556    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:53:59.851819 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.851768    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:59.864285 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.864257    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 17:53:59.865395 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.865378    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal"
Jul 07 17:53:59.879671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:53:59.879644    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 17:54:00.433047 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.433012    2580 apiserver.go:52] "Watching apiserver"
Jul 07 17:54:00.445387 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.445353    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 17:54:00.447517 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.447102    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-nvjjz","kube-system/global-pull-secret-syncer-mslq2","kube-system/konnectivity-agent-8gqxt","openshift-dns/node-resolver-ssnxv","openshift-image-registry/node-ca-j92kl","openshift-multus/multus-additional-cni-plugins-lnsgh","openshift-multus/multus-phkq4","kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d","openshift-cluster-node-tuning-operator/tuned-wmj8c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal","openshift-multus/network-metrics-daemon-8gfg2","openshift-network-diagnostics/network-check-target-vmnjw","openshift-network-operator/iptables-alerter-rhpmq"]
Jul 07 17:54:00.448974 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.448933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.452004 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.451984    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 17:54:00.455701 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.455680    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 17:54:00.455806 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.455755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tkwrm\""
Jul 07 17:54:00.455948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.455920    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.462173 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.462146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.462616 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.462590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.463021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:00.463112 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463037    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 17:54:00.463175 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.463093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:00.463274 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:00.463344 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.463344 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-system-cni-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463455 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-cni-multus\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463455 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-os-release\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463558 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9b5d02e-0f78-4531-a756-d22e736b8e67-cni-binary-copy\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463558 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-socket-dir-parent\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463661 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-k8s-cni-cncf-io\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463661 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-daemon-config\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463661 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-etc-kubernetes\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463811 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-cni-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463811 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463710    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-netns\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463811 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-kubelet\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.463811 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-hostroot\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.464058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-multus-certs\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.464058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.463881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-cnibin\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.464202 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.464180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.464649 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.464621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-cni-bin\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.464731 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.464677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-conf-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.468451 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.468428    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.468610 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.468596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7rbv\" (UniqueName: \"kubernetes.io/projected/f9b5d02e-0f78-4531-a756-d22e736b8e67-kube-api-access-d7rbv\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.468669 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.468657    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.469353 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.469334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.469513 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.469493    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vlwzk\""
Jul 07 17:54:00.469669 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.469652    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.470020 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.470000    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mptkz\""
Jul 07 17:54:00.470112 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.470071    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 17:54:00.470280 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.470262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 17:54:00.470590 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.470564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.472203 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.472183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.472332 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.472222    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9tgc8\""
Jul 07 17:54:00.472439 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.472424    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 17:54:00.472637 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.472618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.473509 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.473488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kqgs8\""
Jul 07 17:54:00.473509 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.473503    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.473818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.473795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.473916 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.473869    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 17:54:00.473916 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.473886    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:00.474220 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.474202    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 17:54:00.475292 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.474983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.475389 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.475350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 17:54:00.476813 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.476793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.478382 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.478360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:00.478476 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.478416    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:00.479072 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479047    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.479170 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479072    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 17:54:00.479366 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 17:54:00.479639 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lww89\""
Jul 07 17:54:00.479639 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479635    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.479867 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479849    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.479968 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8zjgn\""
Jul 07 17:54:00.479968 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.479942    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 17:54:00.480158 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480139    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.480296 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480271    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dg7d8\""
Jul 07 17:54:00.480382 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480314    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4spjk\""
Jul 07 17:54:00.480382 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480327    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.480491 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480428    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 17:54:00.480547 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480527    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 17:54:00.480547 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480537    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 17:54:00.480696 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480652    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 17:54:00.480854 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.480831    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:54:00.500156 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.500130    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rw7nt"
Jul 07 17:54:00.510019 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.509993    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rw7nt"
Jul 07 17:54:00.553315 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.552230    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 17:54:00.568837 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.568801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw7h4\" (UniqueName: \"kubernetes.io/projected/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-kube-api-access-fw7h4\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.568837 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.568834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bb84298-d8ec-4483-a127-fca6ac30e41d-host-slash\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.569075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.568854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.569075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.568880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-cni-netd\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.569075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.568960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovn-node-metrics-cert\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.569075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.568987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-os-release\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9b5d02e-0f78-4531-a756-d22e736b8e67-cni-binary-copy\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569291 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569151    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-os-release\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569291 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-daemon-config\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569291 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-os-release\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.569291 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-cni-bin\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.569456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-tmp-dir\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.569456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysctl-conf\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.569456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-kubelet\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.569671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569499    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-kubelet\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-etc-selinux\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.569671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysctl-d\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.569671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp7fb\" (UniqueName: \"kubernetes.io/projected/6bb84298-d8ec-4483-a127-fca6ac30e41d-kube-api-access-mp7fb\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.569671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-cnibin\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569671 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-conf-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569693    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9b5d02e-0f78-4531-a756-d22e736b8e67-cni-binary-copy\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7rbv\" (UniqueName: \"kubernetes.io/projected/f9b5d02e-0f78-4531-a756-d22e736b8e67-kube-api-access-d7rbv\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569745    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-conf-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcpnt\" (UniqueName: \"kubernetes.io/projected/12ec833e-57b5-4f1d-9606-4d8cd84b387f-kube-api-access-jcpnt\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569839    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-cnibin\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-var-lib-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.569956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-env-overrides\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.569987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eca02787-2090-4af2-a5f9-4a9484e386fc-konnectivity-ca\") pod \"konnectivity-agent-8gqxt\" (UID: \"eca02787-2090-4af2-a5f9-4a9484e386fc\") " pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-log-socket\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovnkube-config\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-socket-dir-parent\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-k8s-cni-cncf-io\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-run-netns\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-etc-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-cni-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570146    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-socket-dir-parent\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570148    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-k8s-cni-cncf-io\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-hostroot\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570200    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-hostroot\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-multus-certs\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cnibin\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.570264 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570267    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-cni-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570272    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-multus-certs\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-cni-bin\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570358    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-cni-bin\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-registration-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-run-ovn-kubernetes\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovnkube-script-lib\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-kubernetes\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-modprobe-d\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eca02787-2090-4af2-a5f9-4a9484e386fc-agent-certs\") pod \"konnectivity-agent-8gqxt\" (UID: \"eca02787-2090-4af2-a5f9-4a9484e386fc\") " pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74f5befa-b459-42f7-806b-2eb48e2fd373-host\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570638    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9b5d02e-0f78-4531-a756-d22e736b8e67-multus-daemon-config\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571058 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-kubelet\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-ovn\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70cf8dba-b3eb-44fd-941d-293797635b91-tmp\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70cf8dba-b3eb-44fd-941d-293797635b91-etc-tuned\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-system-cni-dir\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-systemd\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-node-log\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-sys\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2l66\" (UniqueName: \"kubernetes.io/projected/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-kube-api-access-w2l66\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.570985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-system-cni-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-cni-multus\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/74f5befa-b459-42f7-806b-2eb48e2fd373-serviceca\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-socket-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571052    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-system-cni-dir\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6xgq\" (UniqueName: \"kubernetes.io/projected/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-kube-api-access-x6xgq\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571063    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-var-lib-cni-multus\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.571948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-dbus\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-lib-modules\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-var-lib-kubelet\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-etc-kubernetes\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-device-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-kubelet-config\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-systemd\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-run\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571225    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-etc-kubernetes\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-host\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vntv7\" (UniqueName: \"kubernetes.io/projected/70cf8dba-b3eb-44fd-941d-293797635b91-kube-api-access-vntv7\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-netns\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg74z\" (UniqueName: \"kubernetes.io/projected/74f5befa-b459-42f7-806b-2eb48e2fd373-kube-api-access-dg74z\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571410    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9b5d02e-0f78-4531-a756-d22e736b8e67-host-run-netns\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cni-binary-copy\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-sys-fs\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzfz9\" (UniqueName: \"kubernetes.io/projected/283924a4-b647-43bf-b6df-1e97f8d0fe0e-kube-api-access-mzfz9\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.572689 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysconfig\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.573253 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.571636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bb84298-d8ec-4483-a127-fca6ac30e41d-iptables-alerter-script\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.573253 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.572141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-systemd-units\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.573253 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.572183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-slash\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.573253 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.572222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-hosts-file\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.577964 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.577942    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:00.577964 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.577963    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:00.578161 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.577977    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:00.578161 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.578045    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:01.078027823 +0000 UTC m=+2.968388530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:00.578261 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.578206    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 17:54:00.581958 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.581933    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7rbv\" (UniqueName: \"kubernetes.io/projected/f9b5d02e-0f78-4531-a756-d22e736b8e67-kube-api-access-d7rbv\") pod \"multus-phkq4\" (UID: \"f9b5d02e-0f78-4531-a756-d22e736b8e67\") " pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.628390 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.628349    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:54:00.672503 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-system-cni-dir\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-systemd\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-node-log\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-sys\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672569    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-system-cni-dir\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672603    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-node-log\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2l66\" (UniqueName: \"kubernetes.io/projected/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-kube-api-access-w2l66\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/74f5befa-b459-42f7-806b-2eb48e2fd373-serviceca\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-sys\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672608    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-systemd\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.672703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-socket-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6xgq\" (UniqueName: \"kubernetes.io/projected/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-kube-api-access-x6xgq\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-dbus\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-lib-modules\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-var-lib-kubelet\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-device-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-kubelet-config\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-systemd\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-run\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672864    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-socket-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-host\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vntv7\" (UniqueName: \"kubernetes.io/projected/70cf8dba-b3eb-44fd-941d-293797635b91-kube-api-access-vntv7\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg74z\" (UniqueName: \"kubernetes.io/projected/74f5befa-b459-42f7-806b-2eb48e2fd373-kube-api-access-dg74z\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cni-binary-copy\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-sys-fs\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzfz9\" (UniqueName: \"kubernetes.io/projected/283924a4-b647-43bf-b6df-1e97f8d0fe0e-kube-api-access-mzfz9\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672999    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-systemd\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.673218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysconfig\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bb84298-d8ec-4483-a127-fca6ac30e41d-iptables-alerter-script\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-systemd-units\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-slash\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-hosts-file\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673131    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-lib-modules\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw7h4\" (UniqueName: \"kubernetes.io/projected/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-kube-api-access-fw7h4\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673172    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/74f5befa-b459-42f7-806b-2eb48e2fd373-serviceca\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bb84298-d8ec-4483-a127-fca6ac30e41d-host-slash\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.672976    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-dbus\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673207    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-slash\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673227    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-kubelet-config\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673244    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysconfig\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-run\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673259    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-host\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673232    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bb84298-d8ec-4483-a127-fca6ac30e41d-host-slash\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673272    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-var-lib-kubelet\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673310    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-systemd-units\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674098 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673405    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-device-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673459    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-cni-netd\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673572    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-hosts-file\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovn-node-metrics-cert\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-os-release\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673650    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-cni-netd\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-cni-bin\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673689    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-cni-bin\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-tmp-dir\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673713    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-sys-fs\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysctl-conf\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-etc-selinux\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysctl-d\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp7fb\" (UniqueName: \"kubernetes.io/projected/6bb84298-d8ec-4483-a127-fca6ac30e41d-kube-api-access-mp7fb\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.674954 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcpnt\" (UniqueName: \"kubernetes.io/projected/12ec833e-57b5-4f1d-9606-4d8cd84b387f-kube-api-access-jcpnt\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673872    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysctl-conf\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673879    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bb84298-d8ec-4483-a127-fca6ac30e41d-iptables-alerter-script\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-var-lib-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673908    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cni-binary-copy\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673960    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-tmp-dir\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673987    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-var-lib-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-env-overrides\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eca02787-2090-4af2-a5f9-4a9484e386fc-konnectivity-ca\") pod \"konnectivity-agent-8gqxt\" (UID: \"eca02787-2090-4af2-a5f9-4a9484e386fc\") " pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-log-socket\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovnkube-config\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-sysctl-d\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673990    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-etc-selinux\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-run-netns\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.673752    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-os-release\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-etc-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cnibin\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.675842 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.674198    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.674269    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:01.174251045 +0000 UTC m=+3.064611762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-registration-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-run-ovn-kubernetes\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovnkube-script-lib\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-kubernetes\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674420    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-env-overrides\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-modprobe-d\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674447    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-run-ovn-kubernetes\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eca02787-2090-4af2-a5f9-4a9484e386fc-agent-certs\") pod \"konnectivity-agent-8gqxt\" (UID: \"eca02787-2090-4af2-a5f9-4a9484e386fc\") " pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674666    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674676    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cnibin\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eca02787-2090-4af2-a5f9-4a9484e386fc-konnectivity-ca\") pod \"konnectivity-agent-8gqxt\" (UID: \"eca02787-2090-4af2-a5f9-4a9484e386fc\") " pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.677024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674720    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-log-socket\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674757    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12ec833e-57b5-4f1d-9606-4d8cd84b387f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674779    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.674800    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674825    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-run-netns\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:00.674853    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:01.174836119 +0000 UTC m=+3.065196845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74f5befa-b459-42f7-806b-2eb48e2fd373-host\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674876    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-etc-openvswitch\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674877    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-registration-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674916    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-kubernetes\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674938    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovnkube-script-lib\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.674960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675012    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74f5befa-b459-42f7-806b-2eb48e2fd373-host\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-kubelet\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-ovn\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675066    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/283924a4-b647-43bf-b6df-1e97f8d0fe0e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-host-kubelet\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.677818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675126    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70cf8dba-b3eb-44fd-941d-293797635b91-etc-modprobe-d\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70cf8dba-b3eb-44fd-941d-293797635b91-tmp\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675224    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-run-ovn\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70cf8dba-b3eb-44fd-941d-293797635b91-etc-tuned\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675481    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12ec833e-57b5-4f1d-9606-4d8cd84b387f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.675503    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovnkube-config\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.676568    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-ovn-node-metrics-cert\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.677045    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eca02787-2090-4af2-a5f9-4a9484e386fc-agent-certs\") pod \"konnectivity-agent-8gqxt\" (UID: \"eca02787-2090-4af2-a5f9-4a9484e386fc\") " pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.677695    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70cf8dba-b3eb-44fd-941d-293797635b91-tmp\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.678349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.677920    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70cf8dba-b3eb-44fd-941d-293797635b91-etc-tuned\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.686485 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.686415    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2l66\" (UniqueName: \"kubernetes.io/projected/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-kube-api-access-w2l66\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:00.687144 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.687104    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw7h4\" (UniqueName: \"kubernetes.io/projected/b5f0f3c4-0c42-4df4-a9ad-614e98250cb8-kube-api-access-fw7h4\") pod \"node-resolver-ssnxv\" (UID: \"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8\") " pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.687445 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.687422    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzfz9\" (UniqueName: \"kubernetes.io/projected/283924a4-b647-43bf-b6df-1e97f8d0fe0e-kube-api-access-mzfz9\") pod \"aws-ebs-csi-driver-node-djm7d\" (UID: \"283924a4-b647-43bf-b6df-1e97f8d0fe0e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.688047 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.688022    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6xgq\" (UniqueName: \"kubernetes.io/projected/2c46cd79-3268-4b83-bbef-ffffe0fc9b98-kube-api-access-x6xgq\") pod \"ovnkube-node-nvjjz\" (UID: \"2c46cd79-3268-4b83-bbef-ffffe0fc9b98\") " pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.688948 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.688915    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg74z\" (UniqueName: \"kubernetes.io/projected/74f5befa-b459-42f7-806b-2eb48e2fd373-kube-api-access-dg74z\") pod \"node-ca-j92kl\" (UID: \"74f5befa-b459-42f7-806b-2eb48e2fd373\") " pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.689580 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.689549    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp7fb\" (UniqueName: \"kubernetes.io/projected/6bb84298-d8ec-4483-a127-fca6ac30e41d-kube-api-access-mp7fb\") pod \"iptables-alerter-rhpmq\" (UID: \"6bb84298-d8ec-4483-a127-fca6ac30e41d\") " pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:00.689885 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.689864    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vntv7\" (UniqueName: \"kubernetes.io/projected/70cf8dba-b3eb-44fd-941d-293797635b91-kube-api-access-vntv7\") pod \"tuned-wmj8c\" (UID: \"70cf8dba-b3eb-44fd-941d-293797635b91\") " pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.690078 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.690061    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcpnt\" (UniqueName: \"kubernetes.io/projected/12ec833e-57b5-4f1d-9606-4d8cd84b387f-kube-api-access-jcpnt\") pod \"multus-additional-cni-plugins-lnsgh\" (UID: \"12ec833e-57b5-4f1d-9606-4d8cd84b387f\") " pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.768132 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.768083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phkq4"
Jul 07 17:54:00.780772 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:00.780738    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b5d02e_0f78_4531_a756_d22e736b8e67.slice/crio-98a0664b38dd52f917154f1bac8cf93634e0e5606b36a8668fa715ea4be99c57 WatchSource:0}: Error finding container 98a0664b38dd52f917154f1bac8cf93634e0e5606b36a8668fa715ea4be99c57: Status 404 returned error can't find the container with id 98a0664b38dd52f917154f1bac8cf93634e0e5606b36a8668fa715ea4be99c57
Jul 07 17:54:00.782052 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.782026    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ssnxv"
Jul 07 17:54:00.792294 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.792264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:00.794483 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:00.794448    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f0f3c4_0c42_4df4_a9ad_614e98250cb8.slice/crio-e7958a10036ed5c759b4f801db8186e566bc9a90b15bd1e8771975bf7e4bfd0f WatchSource:0}: Error finding container e7958a10036ed5c759b4f801db8186e566bc9a90b15bd1e8771975bf7e4bfd0f: Status 404 returned error can't find the container with id e7958a10036ed5c759b4f801db8186e566bc9a90b15bd1e8771975bf7e4bfd0f
Jul 07 17:54:00.800940 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.800919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j92kl"
Jul 07 17:54:00.806317 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:00.806283    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca02787_2090_4af2_a5f9_4a9484e386fc.slice/crio-06fe4bb44d5323d30b896918e7c1d2faaaf5169612356f31c391da0dd12627ab WatchSource:0}: Error finding container 06fe4bb44d5323d30b896918e7c1d2faaaf5169612356f31c391da0dd12627ab: Status 404 returned error can't find the container with id 06fe4bb44d5323d30b896918e7c1d2faaaf5169612356f31c391da0dd12627ab
Jul 07 17:54:00.809339 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.809310    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lnsgh"
Jul 07 17:54:00.813797 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:00.813765    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f5befa_b459_42f7_806b_2eb48e2fd373.slice/crio-ec5038737615a46e0971b34b75c294534d484b113ec7fc9a6db2b64854e9adc2 WatchSource:0}: Error finding container ec5038737615a46e0971b34b75c294534d484b113ec7fc9a6db2b64854e9adc2: Status 404 returned error can't find the container with id ec5038737615a46e0971b34b75c294534d484b113ec7fc9a6db2b64854e9adc2
Jul 07 17:54:00.819123 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.819094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d"
Jul 07 17:54:00.819785 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:00.819757    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ec833e_57b5_4f1d_9606_4d8cd84b387f.slice/crio-a08ee182bf6876fa4af75a83f5e52e72c2219fe4dda9d712a50376ea8752a696 WatchSource:0}: Error finding container a08ee182bf6876fa4af75a83f5e52e72c2219fe4dda9d712a50376ea8752a696: Status 404 returned error can't find the container with id a08ee182bf6876fa4af75a83f5e52e72c2219fe4dda9d712a50376ea8752a696
Jul 07 17:54:00.828503 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.828475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:00.836565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.836514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wmj8c"
Jul 07 17:54:00.845566 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:00.845539    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhpmq"
Jul 07 17:54:01.079322 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.079216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:01.079498 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.079397    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:01.079498 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.079419    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:01.079498 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.079433    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:01.079623 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.079499    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:02.079480238 +0000 UTC m=+3.969840948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:01.180132 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.180091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:01.180327 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.180143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:01.180327 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.180275    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:01.180327 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.180276    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:01.180485 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.180343    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:02.180325168 +0000 UTC m=+4.070685870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:01.180485 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:01.180364    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:02.180354258 +0000 UTC m=+4.070714962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:01.512419 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.512360    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:49:00 +0000 UTC" deadline="2027-12-06 22:08:24.027230048 +0000 UTC"
Jul 07 17:54:01.512419 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.512411    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12412h14m22.514821792s"
Jul 07 17:54:01.600063 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.600016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ssnxv" event={"ID":"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8","Type":"ContainerStarted","Data":"e7958a10036ed5c759b4f801db8186e566bc9a90b15bd1e8771975bf7e4bfd0f"}
Jul 07 17:54:01.601358 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.601323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phkq4" event={"ID":"f9b5d02e-0f78-4531-a756-d22e736b8e67","Type":"ContainerStarted","Data":"98a0664b38dd52f917154f1bac8cf93634e0e5606b36a8668fa715ea4be99c57"}
Jul 07 17:54:01.602591 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.602564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerStarted","Data":"a08ee182bf6876fa4af75a83f5e52e72c2219fe4dda9d712a50376ea8752a696"}
Jul 07 17:54:01.603818 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.603784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j92kl" event={"ID":"74f5befa-b459-42f7-806b-2eb48e2fd373","Type":"ContainerStarted","Data":"ec5038737615a46e0971b34b75c294534d484b113ec7fc9a6db2b64854e9adc2"}
Jul 07 17:54:01.605114 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:01.605069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8gqxt" event={"ID":"eca02787-2090-4af2-a5f9-4a9484e386fc","Type":"ContainerStarted","Data":"06fe4bb44d5323d30b896918e7c1d2faaaf5169612356f31c391da0dd12627ab"}
Jul 07 17:54:01.927546 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:01.927503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod283924a4_b647_43bf_b6df_1e97f8d0fe0e.slice/crio-8cc26dd098207d263cd5165155f579ec213097174606a3877b1864f952c537e5 WatchSource:0}: Error finding container 8cc26dd098207d263cd5165155f579ec213097174606a3877b1864f952c537e5: Status 404 returned error can't find the container with id 8cc26dd098207d263cd5165155f579ec213097174606a3877b1864f952c537e5
Jul 07 17:54:01.949866 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:01.949836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c46cd79_3268_4b83_bbef_ffffe0fc9b98.slice/crio-8cafbaa6b6f1664aa2d0d86e29ff2cc8c0689d1267e0333baf6e1124f237331c WatchSource:0}: Error finding container 8cafbaa6b6f1664aa2d0d86e29ff2cc8c0689d1267e0333baf6e1124f237331c: Status 404 returned error can't find the container with id 8cafbaa6b6f1664aa2d0d86e29ff2cc8c0689d1267e0333baf6e1124f237331c
Jul 07 17:54:01.951188 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:01.950954    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70cf8dba_b3eb_44fd_941d_293797635b91.slice/crio-a66e54e9333c24ed0ad4bcdd46deb5cc22df8722de7d6ff08e806e4f72c0d8ae WatchSource:0}: Error finding container a66e54e9333c24ed0ad4bcdd46deb5cc22df8722de7d6ff08e806e4f72c0d8ae: Status 404 returned error can't find the container with id a66e54e9333c24ed0ad4bcdd46deb5cc22df8722de7d6ff08e806e4f72c0d8ae
Jul 07 17:54:01.953210 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:01.952712    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb84298_d8ec_4483_a127_fca6ac30e41d.slice/crio-1fc4cb528d860a46568cba582e4875a86c4f4eb1c6e60449fa88a1ce3fa26f5d WatchSource:0}: Error finding container 1fc4cb528d860a46568cba582e4875a86c4f4eb1c6e60449fa88a1ce3fa26f5d: Status 404 returned error can't find the container with id 1fc4cb528d860a46568cba582e4875a86c4f4eb1c6e60449fa88a1ce3fa26f5d
Jul 07 17:54:02.088370 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.088156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:02.088370 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.088335    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:02.088370 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.088357    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:02.088370 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.088371    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:02.088714 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.088437    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:04.08841857 +0000 UTC m=+5.978779293 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:02.188626 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.188482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:02.188626 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.188539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:02.188849 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.188674    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:02.188849 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.188733    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:04.188719302 +0000 UTC m=+6.079080006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:02.189144 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.189124    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:02.189244 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.189177    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:04.18916021 +0000 UTC m=+6.079520913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:02.512949 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.512889    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:49:00 +0000 UTC" deadline="2027-12-11 20:43:55.79832133 +0000 UTC"
Jul 07 17:54:02.512949 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.512941    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12530h49m53.285385125s"
Jul 07 17:54:02.591898 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.591862    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:02.592116 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.592038    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:02.592491 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.592470    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:02.592600 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.592578    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:02.592678 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.592663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:02.592830 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:02.592756    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:02.617848 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.617473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"8cafbaa6b6f1664aa2d0d86e29ff2cc8c0689d1267e0333baf6e1124f237331c"}
Jul 07 17:54:02.623251 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.623212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d" event={"ID":"283924a4-b647-43bf-b6df-1e97f8d0fe0e","Type":"ContainerStarted","Data":"8cc26dd098207d263cd5165155f579ec213097174606a3877b1864f952c537e5"}
Jul 07 17:54:02.632686 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.632645    2580 generic.go:356] "Generic (PLEG): container finished" podID="afa4c1649d9a92df7d64f13fdc5951de" containerID="7ad07ccf70ba498e185ef4a93cd686459b931a550dd5fe79f715a9f4663a5570" exitCode=0
Jul 07 17:54:02.632856 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.632770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal" event={"ID":"afa4c1649d9a92df7d64f13fdc5951de","Type":"ContainerDied","Data":"7ad07ccf70ba498e185ef4a93cd686459b931a550dd5fe79f715a9f4663a5570"}
Jul 07 17:54:02.638269 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.636924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal" event={"ID":"d0953349d49bbfb3ea71d025821a8f5d","Type":"ContainerStarted","Data":"9d9cf5dadc57b67ce99696136f1e7f72adc240fdc79c59cf7a60a980cd5568c8"}
Jul 07 17:54:02.651510 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.651468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhpmq" event={"ID":"6bb84298-d8ec-4483-a127-fca6ac30e41d","Type":"ContainerStarted","Data":"1fc4cb528d860a46568cba582e4875a86c4f4eb1c6e60449fa88a1ce3fa26f5d"}
Jul 07 17:54:02.659492 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:02.659447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wmj8c" event={"ID":"70cf8dba-b3eb-44fd-941d-293797635b91","Type":"ContainerStarted","Data":"a66e54e9333c24ed0ad4bcdd46deb5cc22df8722de7d6ff08e806e4f72c0d8ae"}
Jul 07 17:54:03.681097 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:03.681056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal" event={"ID":"afa4c1649d9a92df7d64f13fdc5951de","Type":"ContainerStarted","Data":"c8ff145fbd040826c4c73362635c1f316adece1d6f8143e96c0b2ccdabfb25ab"}
Jul 07 17:54:03.701138 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:03.701056    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-243.ec2.internal" podStartSLOduration=3.7010329779999998 podStartE2EDuration="3.701032978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:02.673247454 +0000 UTC m=+4.563608183" watchObservedRunningTime="2026-07-07 17:54:03.701032978 +0000 UTC m=+5.591393699"
Jul 07 17:54:03.701505 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:03.701464    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-243.ec2.internal" podStartSLOduration=3.701448278 podStartE2EDuration="3.701448278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:54:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:03.70122013 +0000 UTC m=+5.591580838" watchObservedRunningTime="2026-07-07 17:54:03.701448278 +0000 UTC m=+5.591809008"
Jul 07 17:54:04.105912 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:04.105816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:04.106113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.106021    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:04.106113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.106047    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:04.106113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.106060    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:04.106318 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.106122    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:08.106101129 +0000 UTC m=+9.996461836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:04.206628 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:04.206588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:04.206825 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:04.206650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:04.206825 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.206811    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:04.206952 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.206877    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:08.206856773 +0000 UTC m=+10.097217485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:04.207361 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.207338    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:04.207461 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.207404    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:08.207384412 +0000 UTC m=+10.097745123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:04.590871 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:04.590065    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:04.590871 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.590215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:04.590871 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:04.590661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:04.590871 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.590755    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:04.591499 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:04.591334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:04.591499 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:04.591453    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:06.591578 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:06.591045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:06.591578 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:06.591205    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:06.591578 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:06.591297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:06.591578 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:06.591403    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:06.591578 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:06.591449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:06.591578 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:06.591538    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:08.143627 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:08.143565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:08.144237 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.143770    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:08.144237 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.143791    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:08.144237 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.143803    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:08.144237 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.143865    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:16.143847819 +0000 UTC m=+18.034208536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:08.244811 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:08.244763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:08.245036 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:08.244829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:08.245036 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.245017    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:08.245155 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.245088    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:16.245071351 +0000 UTC m=+18.135432073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:08.245525 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.245505    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:08.245612 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.245563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:16.245547137 +0000 UTC m=+18.135907839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:08.591775 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:08.591042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:08.591775 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.591164    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:08.591775 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:08.591568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:08.591775 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.591657    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:08.592366 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:08.592204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:08.592366 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:08.592309    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:10.588602 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:10.588565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:10.588602 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:10.588603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:10.589208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:10.588565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:10.589208 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:10.588716    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:10.589208 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:10.588845    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:10.589208 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:10.588957    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:12.589121 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:12.589041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:12.589541 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:12.589042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:12.589541 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:12.589179    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:12.589541 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:12.589198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:12.589541 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:12.589317    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:12.589541 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:12.589407    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:14.589456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:14.589412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:14.589456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:14.589445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:14.589976 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:14.589415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:14.589976 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:14.589581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:14.589976 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:14.589677    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:14.589976 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:14.589755    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:16.205677 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:16.205631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:16.206113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.205774    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:16.206113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.205794    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:16.206113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.205804    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:16.206113 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.205860    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:32.205844244 +0000 UTC m=+34.096204950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:16.306774 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:16.306735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:16.306965 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:16.306830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:16.306965 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.306895    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:16.306965 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.306941    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:16.307076 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.306975    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:32.30695993 +0000 UTC m=+34.197320636 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:16.307076 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.306990    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:32.306984115 +0000 UTC m=+34.197344816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:16.589217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:16.589135    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:16.589217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:16.589190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:16.589217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:16.589200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:16.589485 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.589288    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:16.589485 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.589442    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:16.589574 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:16.589518    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:18.589756 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:18.589724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:18.590225 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:18.589851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:18.590225 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:18.589872    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:18.590225 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:18.589927    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:18.590225 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:18.589955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:18.590225 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:18.590014    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:20.589272 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:20.589236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:20.589719 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:20.589241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:20.589719 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:20.589392    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:20.589719 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:20.589490    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:20.589719 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:20.589236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:20.589719 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:20.589591    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:21.717585 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.717221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wmj8c" event={"ID":"70cf8dba-b3eb-44fd-941d-293797635b91","Type":"ContainerStarted","Data":"10dd2db96452d23e45a702484493ee45c6ddfc3bf369a87b120dc1ec6b865ed0"}
Jul 07 17:54:21.718747 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.718725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8gqxt" event={"ID":"eca02787-2090-4af2-a5f9-4a9484e386fc","Type":"ContainerStarted","Data":"bb39868b43ef8af444d29dbd27d3122209e037e465769af8de5602ed09bca793"}
Jul 07 17:54:21.720194 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.720172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ssnxv" event={"ID":"b5f0f3c4-0c42-4df4-a9ad-614e98250cb8","Type":"ContainerStarted","Data":"9af1d4e2c633623810d54dc6756e545434a1d624e2bbfbda55cc955a99e0edce"}
Jul 07 17:54:21.723699 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.723676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"f39c6c90bceb884bddaa19ef38b082750dd14b54ab8590aab14254aef209414b"}
Jul 07 17:54:21.723807 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.723756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"72e07a563030f979d617a23fbefd94387dc52f34943fdab4c7ee986f93bc2567"}
Jul 07 17:54:21.723869 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.723808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"f43ed16de99735b15b20abf0646d2c07360621d7a8f43bbeeedbfd909744ba6c"}
Jul 07 17:54:21.723869 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.723856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"df3e6fd526bc54adbd873057aa2df4434361906748b558fdd0113f73d22927b0"}
Jul 07 17:54:21.724002 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.723924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"08d3701c4ba06cbbfcf1e0d0a4f20ba3f1552f6e512e657530ac5b7aff8eadfe"}
Jul 07 17:54:21.724002 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.723972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"eb0ab3bc258d5409b1fff8893254f5a8c1766d45e07b08965db8a2687b18f767"}
Jul 07 17:54:21.724879 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.724857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d" event={"ID":"283924a4-b647-43bf-b6df-1e97f8d0fe0e","Type":"ContainerStarted","Data":"03bc1c454d860691948aac929e8e97eb65c3bb41083591f88f1f00a4cc271a1a"}
Jul 07 17:54:21.726203 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.726179    2580 generic.go:356] "Generic (PLEG): container finished" podID="12ec833e-57b5-4f1d-9606-4d8cd84b387f" containerID="5aa172d6f4c9b88a74679e32447cda7cf12b26349481e9bf120e29b537dd7992" exitCode=0
Jul 07 17:54:21.726292 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.726255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerDied","Data":"5aa172d6f4c9b88a74679e32447cda7cf12b26349481e9bf120e29b537dd7992"}
Jul 07 17:54:21.733631 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.733607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j92kl" event={"ID":"74f5befa-b459-42f7-806b-2eb48e2fd373","Type":"ContainerStarted","Data":"52ad0da3c10e39ecc02357f3f4b8ff0ad4a538485575de74ba3e79a55223fa26"}
Jul 07 17:54:21.735319 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.735271    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wmj8c" podStartSLOduration=4.085585174 podStartE2EDuration="22.735254452s" totalImagesPullingTime="18.649669278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:21.735125008 +0000 UTC m=+23.625485744" watchObservedRunningTime="2026-07-07 17:54:21.735254452 +0000 UTC m=+23.625615177"
Jul 07 17:54:21.735420 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.735389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phkq4" event={"ID":"f9b5d02e-0f78-4531-a756-d22e736b8e67","Type":"ContainerStarted","Data":"b78fa1dac9d5a4572d3cd1fc34a692a184db8abe78455f9fc4178fb85034d855"}
Jul 07 17:54:21.751537 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.751493    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ssnxv" podStartSLOduration=2.953674312 podStartE2EDuration="22.751479022s" totalImagesPullingTime="19.79780471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:21.751334897 +0000 UTC m=+23.641695621" watchObservedRunningTime="2026-07-07 17:54:21.751479022 +0000 UTC m=+23.641839745"
Jul 07 17:54:21.789949 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.789877    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8gqxt" podStartSLOduration=3.001221294 podStartE2EDuration="22.78986002s" totalImagesPullingTime="19.788638726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:21.773771768 +0000 UTC m=+23.664132491" watchObservedRunningTime="2026-07-07 17:54:21.78986002 +0000 UTC m=+23.680220744"
Jul 07 17:54:21.790126 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.790014    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j92kl" podStartSLOduration=3.029191768 podStartE2EDuration="22.790008056s" totalImagesPullingTime="19.760816288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:21.789551328 +0000 UTC m=+23.679912069" watchObservedRunningTime="2026-07-07 17:54:21.790008056 +0000 UTC m=+23.680368779"
Jul 07 17:54:21.838078 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:21.838034    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-phkq4" podStartSLOduration=2.968712817 podStartE2EDuration="22.838017886s" totalImagesPullingTime="19.869305069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:21.837386476 +0000 UTC m=+23.727747206" watchObservedRunningTime="2026-07-07 17:54:21.838017886 +0000 UTC m=+23.728378610"
Jul 07 17:54:22.304817 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.304787    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 17:54:22.525080 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.524932    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T17:54:22.30481115Z","UUID":"5cc7428f-1b90-4aa7-9348-1a470298c6b8","Handler":null,"Name":"","Endpoint":""}
Jul 07 17:54:22.526552 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.526535    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 17:54:22.526660 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.526576    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 17:54:22.589471 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.589438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:22.589676 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.589438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:22.589676 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:22.589544    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:22.589676 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.589570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:22.589841 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:22.589681    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:22.589841 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:22.589767    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:22.739446 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.739406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhpmq" event={"ID":"6bb84298-d8ec-4483-a127-fca6ac30e41d","Type":"ContainerStarted","Data":"38dbece605ae10abe7ebd1574ea2f16e475c9eadd082d33ac4bd1ba5624cb893"}
Jul 07 17:54:22.744071 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.744038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d" event={"ID":"283924a4-b647-43bf-b6df-1e97f8d0fe0e","Type":"ContainerStarted","Data":"2e99350a482a086a08b059df127a5802b8be8aa43c7b8ba74602f3479dfa1297"}
Jul 07 17:54:22.796953 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.796883    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:22.797550 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.797529    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:22.815405 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:22.815345    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rhpmq" podStartSLOduration=5.171261058 podStartE2EDuration="23.81532891s" totalImagesPullingTime="18.644067852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:22.765746378 +0000 UTC m=+24.656107102" watchObservedRunningTime="2026-07-07 17:54:22.81532891 +0000 UTC m=+24.705689634"
Jul 07 17:54:23.751204 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:23.751170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"90f138c24d8f458263a348acea7d7a5187b569542edbd500064fefd1a4a762d4"}
Jul 07 17:54:23.752188 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:23.751343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:23.752188 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:23.751786    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8gqxt"
Jul 07 17:54:24.589305 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:24.589074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:24.589518 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:24.589094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:24.589518 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:24.589406    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:24.589518 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:24.589144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:24.589518 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:24.589509    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:24.589727 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:24.589556    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:24.755365 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:24.755326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d" event={"ID":"283924a4-b647-43bf-b6df-1e97f8d0fe0e","Type":"ContainerStarted","Data":"8f10af43b2075d78a2a1473a9109a3fee3e81652d1b1c03f452fdbee88776ab9"}
Jul 07 17:54:24.776657 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:24.776610    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm7d" podStartSLOduration=4.459615853 podStartE2EDuration="25.776593919s" totalImagesPullingTime="21.316978066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:24.776522301 +0000 UTC m=+26.666883028" watchObservedRunningTime="2026-07-07 17:54:24.776593919 +0000 UTC m=+26.666954642"
Jul 07 17:54:25.762480 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:25.762387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" event={"ID":"2c46cd79-3268-4b83-bbef-ffffe0fc9b98","Type":"ContainerStarted","Data":"3d082af299d8e2bc98d2da165a6f7ae0041a0755e794510e4ccfc6ea8fbcae67"}
Jul 07 17:54:25.796505 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:25.796438    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz" podStartSLOduration=8.07835954 podStartE2EDuration="26.796422948s" totalImagesPullingTime="18.718063408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:25.796313089 +0000 UTC m=+27.686673813" watchObservedRunningTime="2026-07-07 17:54:25.796422948 +0000 UTC m=+27.686783673"
Jul 07 17:54:26.588816 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.588513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:26.588984 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.588516    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:26.588984 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:26.588859    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:26.588984 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.588566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:26.588984 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:26.588937    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:26.589165 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:26.589026    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:26.765485 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.765446    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:26.765863 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.765568    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:26.765863 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.765624    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:26.789930 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.789433    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:26.791239 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:26.791213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:54:27.768599 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.768560    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerStarted","Data":"973ddf30a04b3edd508898c63f309dfa8ccde3170243e30fc7317b0afde6c8c0"}
Jul 07 17:54:27.910344 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.910264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mslq2"]
Jul 07 17:54:27.910500 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.910378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:27.910500 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:27.910470    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:27.913035 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.912999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8gfg2"]
Jul 07 17:54:27.913149 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.913116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:27.913216 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:27.913198    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:27.919087 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.919053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vmnjw"]
Jul 07 17:54:27.919208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:27.919184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:27.919303 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:27.919284    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:28.772557 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:28.772520    2580 generic.go:356] "Generic (PLEG): container finished" podID="12ec833e-57b5-4f1d-9606-4d8cd84b387f" containerID="973ddf30a04b3edd508898c63f309dfa8ccde3170243e30fc7317b0afde6c8c0" exitCode=0
Jul 07 17:54:28.773053 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:28.772588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerDied","Data":"973ddf30a04b3edd508898c63f309dfa8ccde3170243e30fc7317b0afde6c8c0"}
Jul 07 17:54:29.589041 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:29.589008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:29.589178 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:29.589008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:29.589178 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:29.589114    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:29.589269 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:29.589206    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:29.589269 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:29.589008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:29.589355 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:29.589282    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:29.776752 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:29.776722    2580 generic.go:356] "Generic (PLEG): container finished" podID="12ec833e-57b5-4f1d-9606-4d8cd84b387f" containerID="6b1beb9cfce67dcd8995bcd176438aeed55e24e31ad9148a924774b93dc8f8bb" exitCode=0
Jul 07 17:54:29.777154 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:29.776766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerDied","Data":"6b1beb9cfce67dcd8995bcd176438aeed55e24e31ad9148a924774b93dc8f8bb"}
Jul 07 17:54:30.782013 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:30.781764    2580 generic.go:356] "Generic (PLEG): container finished" podID="12ec833e-57b5-4f1d-9606-4d8cd84b387f" containerID="10d1fa91a0f3b5a3d4f311249321115e1fe3c245a82dafdd751a3f3846213e98" exitCode=0
Jul 07 17:54:30.782013 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:30.781848    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerDied","Data":"10d1fa91a0f3b5a3d4f311249321115e1fe3c245a82dafdd751a3f3846213e98"}
Jul 07 17:54:31.589063 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:31.589028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:31.589063 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:31.589058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:31.589263 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:31.589028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:31.589263 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:31.589163    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vmnjw" podUID="6ccd0fd1-9356-45e2-97df-f74f8216fa29"
Jul 07 17:54:31.589320 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:31.589290    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mslq2" podUID="0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b"
Jul 07 17:54:31.589392 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:31.589375    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8gfg2" podUID="5f1bba2b-09b2-4a4f-ab55-506d6180ecf8"
Jul 07 17:54:32.223701 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:32.223618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:32.224191 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.223805    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:54:32.224191 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.223829    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:54:32.224191 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.223843    2580 projected.go:196] Error preparing data for projected volume kube-api-access-hzbll for pod openshift-network-diagnostics/network-check-target-vmnjw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:32.224191 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.223931    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll podName:6ccd0fd1-9356-45e2-97df-f74f8216fa29 nodeName:}" failed. No retries permitted until 2026-07-07 17:55:04.223891124 +0000 UTC m=+66.114251828 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hzbll" (UniqueName: "kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll") pod "network-check-target-vmnjw" (UID: "6ccd0fd1-9356-45e2-97df-f74f8216fa29") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:54:32.324725 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:32.324686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:32.324900 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:32.324754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:32.324900 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.324828    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:32.324900 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.324832    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:32.324900 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.324889    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret podName:0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b nodeName:}" failed. No retries permitted until 2026-07-07 17:55:04.324874286 +0000 UTC m=+66.215235018 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret") pod "global-pull-secret-syncer-mslq2" (UID: "0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:54:32.325062 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:32.324923    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:55:04.32489742 +0000 UTC m=+66.215258122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:54:33.431487 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.431408    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-243.ec2.internal" event="NodeReady"
Jul 07 17:54:33.431822 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.431522    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 17:54:33.473087 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.473054    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-769d455b8b-nxxxc"]
Jul 07 17:54:33.474797 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.474781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.477555 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.477535    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c44kt\""
Jul 07 17:54:33.477661 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.477562    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 17:54:33.477774 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.477759    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 17:54:33.477835 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.477790    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 17:54:33.482934 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.482898    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 17:54:33.491333 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.491306    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-769d455b8b-nxxxc"]
Jul 07 17:54:33.493421 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.493398    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-662vr"]
Jul 07 17:54:33.495341 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.495324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.500468 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.500448    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 17:54:33.500753 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.500735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dtpjm\""
Jul 07 17:54:33.500753 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.500818    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 17:54:33.502126 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.502107    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c2z5b"]
Jul 07 17:54:33.503658 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.503643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.506244 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.506229    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 17:54:33.506470 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.506458    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 17:54:33.506525 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.506466    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 17:54:33.506525 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.506479    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f6qmm\""
Jul 07 17:54:33.508308 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.508290    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-662vr"]
Jul 07 17:54:33.514937 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.514885    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c2z5b"]
Jul 07 17:54:33.588895 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.588856    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:54:33.588895 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.588887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:54:33.589158 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.588923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:54:33.592073 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.592049    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 17:54:33.592180 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.592152    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 17:54:33.592250 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.592049    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g4wsg\""
Jul 07 17:54:33.592385 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.592369    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n28xq\""
Jul 07 17:54:33.592445 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.592431    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 17:54:33.592568 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.592554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 17:54:33.632219 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632376 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.632376 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01c155b4-3196-4c5e-9c49-c183312ffc68-cert\") pod \"ingress-canary-c2z5b\" (UID: \"01c155b4-3196-4c5e-9c49-c183312ffc68\") " pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.632376 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-image-registry-private-configuration\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632376 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-config-volume\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.632548 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h8mf\" (UniqueName: \"kubernetes.io/projected/01c155b4-3196-4c5e-9c49-c183312ffc68-kube-api-access-8h8mf\") pod \"ingress-canary-c2z5b\" (UID: \"01c155b4-3196-4c5e-9c49-c183312ffc68\") " pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.632548 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-certificates\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632548 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lc8p\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-kube-api-access-4lc8p\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632548 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa7404ee-8f82-4389-9523-041ad682e3e2-ca-trust-extracted\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-tmp-dir\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.632679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-trusted-ca\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4vmj\" (UniqueName: \"kubernetes.io/projected/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-kube-api-access-p4vmj\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.632768 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-installation-pull-secrets\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.632768 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.632725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-bound-sa-token\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.671854 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.671817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gqxt_eca02787-2090-4af2-a5f9-4a9484e386fc/konnectivity-agent/0.log" limit=1024
Jul 07 17:54:33.733989 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.733891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.733989 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.733960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.733993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01c155b4-3196-4c5e-9c49-c183312ffc68-cert\") pod \"ingress-canary-c2z5b\" (UID: \"01c155b4-3196-4c5e-9c49-c183312ffc68\") " pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-image-registry-private-configuration\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-config-volume\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:33.734056    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:33.734073    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:33.734133    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:34.23411115 +0000 UTC m=+36.124471857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:54:33.734186 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:33.734156    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h8mf\" (UniqueName: \"kubernetes.io/projected/01c155b4-3196-4c5e-9c49-c183312ffc68-kube-api-access-8h8mf\") pod \"ingress-canary-c2z5b\" (UID: \"01c155b4-3196-4c5e-9c49-c183312ffc68\") " pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:33.734220    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:34.234202986 +0000 UTC m=+36.124563687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-certificates\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lc8p\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-kube-api-access-4lc8p\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa7404ee-8f82-4389-9523-041ad682e3e2-ca-trust-extracted\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-tmp-dir\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-trusted-ca\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4vmj\" (UniqueName: \"kubernetes.io/projected/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-kube-api-access-p4vmj\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-installation-pull-secrets\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.734512 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-bound-sa-token\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.735074 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734694    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-config-volume\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.735074 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.734727    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-tmp-dir\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.735074 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.735062    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa7404ee-8f82-4389-9523-041ad682e3e2-ca-trust-extracted\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.735295 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.735249    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-certificates\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.735695 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.735673    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-trusted-ca\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.738455 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.738426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-installation-pull-secrets\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.738547 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.738456    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01c155b4-3196-4c5e-9c49-c183312ffc68-cert\") pod \"ingress-canary-c2z5b\" (UID: \"01c155b4-3196-4c5e-9c49-c183312ffc68\") " pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.738547 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.738458    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-image-registry-private-configuration\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.742743 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.742699    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h8mf\" (UniqueName: \"kubernetes.io/projected/01c155b4-3196-4c5e-9c49-c183312ffc68-kube-api-access-8h8mf\") pod \"ingress-canary-c2z5b\" (UID: \"01c155b4-3196-4c5e-9c49-c183312ffc68\") " pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.744557 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.744530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-bound-sa-token\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.745213 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.745195    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4vmj\" (UniqueName: \"kubernetes.io/projected/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-kube-api-access-p4vmj\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:33.745360 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.745344    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lc8p\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-kube-api-access-4lc8p\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:33.771454 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.771419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8gqxt_eca02787-2090-4af2-a5f9-4a9484e386fc/konnectivity-agent/0.log" limit=1024
Jul 07 17:54:33.814137 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.814094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c2z5b"
Jul 07 17:54:33.985641 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:33.985566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c2z5b"]
Jul 07 17:54:33.990237 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:54:33.990201    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c155b4_3196_4c5e_9c49_c183312ffc68.slice/crio-3d9940998d610df29d4b9e8695d65c648e5c130c84c42b2c71e961ad7d4f9d06 WatchSource:0}: Error finding container 3d9940998d610df29d4b9e8695d65c648e5c130c84c42b2c71e961ad7d4f9d06: Status 404 returned error can't find the container with id 3d9940998d610df29d4b9e8695d65c648e5c130c84c42b2c71e961ad7d4f9d06
Jul 07 17:54:34.237636 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:34.237548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:34.237636 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:34.237586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:34.237863 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:34.237705    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:54:34.237863 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:34.237712    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:54:34.237863 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:34.237731    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:54:34.237863 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:34.237766    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:35.237747128 +0000 UTC m=+37.128107831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:54:34.237863 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:34.237782    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:35.237774025 +0000 UTC m=+37.128134727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:54:34.792626 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:34.792433    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c2z5b" event={"ID":"01c155b4-3196-4c5e-9c49-c183312ffc68","Type":"ContainerStarted","Data":"3d9940998d610df29d4b9e8695d65c648e5c130c84c42b2c71e961ad7d4f9d06"}
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:35.245593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:35.245642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:35.245727    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:35.245747    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:35.245758    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:35.245800    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:37.245785742 +0000 UTC m=+39.136146444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:54:35.245808 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:35.245820    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:37.245809566 +0000 UTC m=+39.136170268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:54:37.263755 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:37.263732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:37.264047 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:37.263764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:37.264047 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:37.263871    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:54:37.264047 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:37.263878    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:54:37.264047 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:37.263894    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:54:37.264047 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:37.263945    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:41.263930763 +0000 UTC m=+43.154291473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:54:37.264047 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:37.263959    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:41.26395311 +0000 UTC m=+43.154313812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:54:37.804294 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:37.804206    2580 generic.go:356] "Generic (PLEG): container finished" podID="12ec833e-57b5-4f1d-9606-4d8cd84b387f" containerID="3ebf7a9154e5af82f9a99f5c1b3ca2b034069e30f6fe60743ef80e39888bad86" exitCode=0
Jul 07 17:54:37.804471 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:37.804296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerDied","Data":"3ebf7a9154e5af82f9a99f5c1b3ca2b034069e30f6fe60743ef80e39888bad86"}
Jul 07 17:54:37.806018 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:37.805982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c2z5b" event={"ID":"01c155b4-3196-4c5e-9c49-c183312ffc68","Type":"ContainerStarted","Data":"61a9cfa95ed838a867d3114f2bd92590743a2b5a7dd769549f6ffd27730ac893"}
Jul 07 17:54:37.885248 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:37.885196    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c2z5b" podStartSLOduration=1.269565331 podStartE2EDuration="4.885179389s" totalImagesPullingTime="3.615614058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:37.884413583 +0000 UTC m=+39.774774307" watchObservedRunningTime="2026-07-07 17:54:37.885179389 +0000 UTC m=+39.775540091"
Jul 07 17:54:38.811487 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:38.811450    2580 generic.go:356] "Generic (PLEG): container finished" podID="12ec833e-57b5-4f1d-9606-4d8cd84b387f" containerID="82fc7eeea825de3701d406a49cb45931c56fcf09cf289792253a54c1684ad3a9" exitCode=0
Jul 07 17:54:38.811944 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:38.811546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerDied","Data":"82fc7eeea825de3701d406a49cb45931c56fcf09cf289792253a54c1684ad3a9"}
Jul 07 17:54:39.816767 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:39.816727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" event={"ID":"12ec833e-57b5-4f1d-9606-4d8cd84b387f","Type":"ContainerStarted","Data":"07fb929978fc21681d8167eddac74517623f6f822257e231e8d843127b929b69"}
Jul 07 17:54:39.844364 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:39.844312    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lnsgh" podStartSLOduration=7.328641767 podStartE2EDuration="40.844295519s" totalImagesPullingTime="33.515653752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:54:39.84216317 +0000 UTC m=+41.732523891" watchObservedRunningTime="2026-07-07 17:54:39.844295519 +0000 UTC m=+41.734656242"
Jul 07 17:54:41.296077 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:41.296032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:41.296077 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:41.296079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:41.296499 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:41.296174    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:54:41.296499 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:41.296179    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:54:41.296499 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:41.296196    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:54:41.296499 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:41.296235    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:54:49.29622173 +0000 UTC m=+51.186582431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:54:41.296499 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:41.296250    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:54:49.296243748 +0000 UTC m=+51.186604449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:54:49.361650 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:49.361606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:54:49.362130 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:49.361664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:54:49.362130 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:49.361760    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:54:49.362130 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:49.361784    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:54:49.362130 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:49.361767    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:54:49.362130 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:49.361848    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:55:05.361831047 +0000 UTC m=+67.252191749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:54:49.362130 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:54:49.361964    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:55:05.361890196 +0000 UTC m=+67.252250902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:54:58.789584 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:54:58.789550    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nvjjz"
Jul 07 17:55:00.436727 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.436691    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v78vv"]
Jul 07 17:55:00.440263 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.440241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.442760 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.442734    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 17:55:00.442892 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.442882    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 17:55:00.443862 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.443844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 17:55:00.443956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.443852    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 17:55:00.443956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.443925    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5jtj4\""
Jul 07 17:55:00.455861 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.455827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v78vv"]
Jul 07 17:55:00.543895 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.543855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljk9c\" (UniqueName: \"kubernetes.io/projected/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-api-access-ljk9c\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.544101 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.543951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9f6b888-066a-498a-afab-6da1a3ab8e9c-crio-socket\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.544101 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.543970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9f6b888-066a-498a-afab-6da1a3ab8e9c-data-volume\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.544101 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.543990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.544101 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.544009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.644969 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.644939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9f6b888-066a-498a-afab-6da1a3ab8e9c-crio-socket\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.644969 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.644970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9f6b888-066a-498a-afab-6da1a3ab8e9c-data-volume\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.645217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.644995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.645217 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:00.645071    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:55:00.645217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.645070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.645217 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.645084    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a9f6b888-066a-498a-afab-6da1a3ab8e9c-crio-socket\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.645217 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:00.645163    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:01.145137485 +0000 UTC m=+63.035498204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : configmap "kube-rbac-proxy" not found
Jul 07 17:55:00.645217 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:00.645189    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:00.645438 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:00.645233    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:01.145220668 +0000 UTC m=+63.035581370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:00.645438 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.645258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljk9c\" (UniqueName: \"kubernetes.io/projected/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-api-access-ljk9c\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.645438 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.645330    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9f6b888-066a-498a-afab-6da1a3ab8e9c-data-volume\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:00.657465 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:00.657438    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljk9c\" (UniqueName: \"kubernetes.io/projected/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-api-access-ljk9c\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:01.148502 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:01.148464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:01.148502 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:01.148505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:01.148727 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:01.148589    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:55:01.148727 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:01.148676    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:02.148660573 +0000 UTC m=+64.039021275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : configmap "kube-rbac-proxy" not found
Jul 07 17:55:01.148727 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:01.148686    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:01.148825 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:01.148775    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:02.148762044 +0000 UTC m=+64.039122746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:02.157831 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:02.157788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:02.157831 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:02.157830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:02.158294 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:02.157926    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:55:02.158294 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:02.157955    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:02.158294 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:02.157994    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:04.157978652 +0000 UTC m=+66.048339355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : configmap "kube-rbac-proxy" not found
Jul 07 17:55:02.158294 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:02.158009    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:04.158002779 +0000 UTC m=+66.048363481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:03.069933 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.069878    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-k2b7d"]
Jul 07 17:55:03.072837 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.072811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.076839 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.076818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5d8ks\""
Jul 07 17:55:03.077423 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.077400    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 17:55:03.077715 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.077701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 17:55:03.077770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.077704    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 17:55:03.077814 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.077727    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 17:55:03.083498 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.083473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-k2b7d"]
Jul 07 17:55:03.266807 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.266770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0316fb25-9cde-48bf-be77-ee78023e01d0-signing-cabundle\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.267223 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.266839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0316fb25-9cde-48bf-be77-ee78023e01d0-signing-key\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.267223 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.266887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx8jv\" (UniqueName: \"kubernetes.io/projected/0316fb25-9cde-48bf-be77-ee78023e01d0-kube-api-access-xx8jv\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.367643 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.367604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0316fb25-9cde-48bf-be77-ee78023e01d0-signing-key\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.367837 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.367672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx8jv\" (UniqueName: \"kubernetes.io/projected/0316fb25-9cde-48bf-be77-ee78023e01d0-kube-api-access-xx8jv\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.367837 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.367815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0316fb25-9cde-48bf-be77-ee78023e01d0-signing-cabundle\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.368415 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.368393    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0316fb25-9cde-48bf-be77-ee78023e01d0-signing-cabundle\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.370315 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.370295    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0316fb25-9cde-48bf-be77-ee78023e01d0-signing-key\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.379086 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.379056    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx8jv\" (UniqueName: \"kubernetes.io/projected/0316fb25-9cde-48bf-be77-ee78023e01d0-kube-api-access-xx8jv\") pod \"service-ca-758f599c5-k2b7d\" (UID: \"0316fb25-9cde-48bf-be77-ee78023e01d0\") " pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.381880 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.381852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-k2b7d"
Jul 07 17:55:03.426413 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.426387    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ssnxv_b5f0f3c4-0c42-4df4-a9ad-614e98250cb8/dns-node-resolver/0.log"
Jul 07 17:55:03.515038 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.515006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-k2b7d"]
Jul 07 17:55:03.518281 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:03.518251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0316fb25_9cde_48bf_be77_ee78023e01d0.slice/crio-59228fbdc6a47abf003079539fd737f91b9f35e62d21db55e38d16ea52165c1f WatchSource:0}: Error finding container 59228fbdc6a47abf003079539fd737f91b9f35e62d21db55e38d16ea52165c1f: Status 404 returned error can't find the container with id 59228fbdc6a47abf003079539fd737f91b9f35e62d21db55e38d16ea52165c1f
Jul 07 17:55:03.872627 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:03.872595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-k2b7d" event={"ID":"0316fb25-9cde-48bf-be77-ee78023e01d0","Type":"ContainerStarted","Data":"59228fbdc6a47abf003079539fd737f91b9f35e62d21db55e38d16ea52165c1f"}
Jul 07 17:55:04.174287 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.174194    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:04.174287 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.174235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:04.174479 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:04.174311    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:55:04.174479 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:04.174381    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:08.174362708 +0000 UTC m=+70.064723412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : configmap "kube-rbac-proxy" not found
Jul 07 17:55:04.174479 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:04.174417    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:04.174479 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:04.174467    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:08.174452856 +0000 UTC m=+70.064813559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:04.226081 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.226046    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j92kl_74f5befa-b459-42f7-806b-2eb48e2fd373/node-ca/0.log"
Jul 07 17:55:04.275458 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.275419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:55:04.278680 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.278644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 17:55:04.289392 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.289363    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 17:55:04.299872 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.299833    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzbll\" (UniqueName: \"kubernetes.io/projected/6ccd0fd1-9356-45e2-97df-f74f8216fa29-kube-api-access-hzbll\") pod \"network-check-target-vmnjw\" (UID: \"6ccd0fd1-9356-45e2-97df-f74f8216fa29\") " pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:55:04.375952 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.375917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:55:04.376103 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.376027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:55:04.378919 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.378870    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 17:55:04.378919 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.378898    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 17:55:04.387151 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:04.387117    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 17:55:04.387332 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:04.387209    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs podName:5f1bba2b-09b2-4a4f-ab55-506d6180ecf8 nodeName:}" failed. No retries permitted until 2026-07-07 17:56:08.387184446 +0000 UTC m=+130.277545151 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs") pod "network-metrics-daemon-8gfg2" (UID: "5f1bba2b-09b2-4a4f-ab55-506d6180ecf8") : secret "metrics-daemon-secret" not found
Jul 07 17:55:04.389860 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.389832    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b-original-pull-secret\") pod \"global-pull-secret-syncer-mslq2\" (UID: \"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b\") " pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:55:04.503959 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.503876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n28xq\""
Jul 07 17:55:04.508330 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.508301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mslq2"
Jul 07 17:55:04.511614 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.511588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:55:04.678260 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.678228    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vmnjw"]
Jul 07 17:55:04.681494 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:04.681468    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ccd0fd1_9356_45e2_97df_f74f8216fa29.slice/crio-71a3db1a2e08b7c9bb59c4772d8c941e4a2ac6701d8d30503c73339ed7e1a9ef WatchSource:0}: Error finding container 71a3db1a2e08b7c9bb59c4772d8c941e4a2ac6701d8d30503c73339ed7e1a9ef: Status 404 returned error can't find the container with id 71a3db1a2e08b7c9bb59c4772d8c941e4a2ac6701d8d30503c73339ed7e1a9ef
Jul 07 17:55:04.692362 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.692332    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mslq2"]
Jul 07 17:55:04.696531 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:04.696504    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0f0ec3_356d_4bdd_b3e3_d434cb600b7b.slice/crio-8f78c6e030cd246c915f4038d2ee47e0205b7c6d6c52aa0cee214d3ed0536d82 WatchSource:0}: Error finding container 8f78c6e030cd246c915f4038d2ee47e0205b7c6d6c52aa0cee214d3ed0536d82: Status 404 returned error can't find the container with id 8f78c6e030cd246c915f4038d2ee47e0205b7c6d6c52aa0cee214d3ed0536d82
Jul 07 17:55:04.826427 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.826340    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c2z5b_01c155b4-3196-4c5e-9c49-c183312ffc68/serve-healthcheck-canary/0.log"
Jul 07 17:55:04.876423 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.876379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vmnjw" event={"ID":"6ccd0fd1-9356-45e2-97df-f74f8216fa29","Type":"ContainerStarted","Data":"71a3db1a2e08b7c9bb59c4772d8c941e4a2ac6701d8d30503c73339ed7e1a9ef"}
Jul 07 17:55:04.877450 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:04.877426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mslq2" event={"ID":"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b","Type":"ContainerStarted","Data":"8f78c6e030cd246c915f4038d2ee47e0205b7c6d6c52aa0cee214d3ed0536d82"}
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:05.386476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:05.386539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:05.386633    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:05.386655    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769d455b8b-nxxxc: secret "image-registry-tls" not found
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:05.386715    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:05.386725    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls podName:fa7404ee-8f82-4389-9523-041ad682e3e2 nodeName:}" failed. No retries permitted until 2026-07-07 17:55:37.386701097 +0000 UTC m=+99.277061822 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls") pod "image-registry-769d455b8b-nxxxc" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2") : secret "image-registry-tls" not found
Jul 07 17:55:05.386793 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:05.386764    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls podName:363e7a50-bf3c-4a89-bb95-8949a1f64c0b nodeName:}" failed. No retries permitted until 2026-07-07 17:55:37.386749012 +0000 UTC m=+99.277109734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls") pod "dns-default-662vr" (UID: "363e7a50-bf3c-4a89-bb95-8949a1f64c0b") : secret "dns-default-metrics-tls" not found
Jul 07 17:55:06.884573 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:06.884529    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-k2b7d" event={"ID":"0316fb25-9cde-48bf-be77-ee78023e01d0","Type":"ContainerStarted","Data":"f076b5c5cb425b631051dd5513a21f7c2b5100880f39f94f2ff9806c222c5612"}
Jul 07 17:55:06.910257 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:06.910190    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-k2b7d" podStartSLOduration=1.348034064 podStartE2EDuration="3.910170717s" totalImagesPullingTime="2.562136653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:06.908892923 +0000 UTC m=+68.799253651" watchObservedRunningTime="2026-07-07 17:55:06.910170717 +0000 UTC m=+68.800531442"
Jul 07 17:55:08.211953 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:08.211394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:08.211953 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:08.211454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:08.211953 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:08.211619    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:08.211953 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:08.211694    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:16.21167533 +0000 UTC m=+78.102036036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:08.212619 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:08.212040    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:55:08.212619 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:08.212126    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:16.212105668 +0000 UTC m=+78.102466376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : configmap "kube-rbac-proxy" not found
Jul 07 17:55:10.898920 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:10.898867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vmnjw" event={"ID":"6ccd0fd1-9356-45e2-97df-f74f8216fa29","Type":"ContainerStarted","Data":"e18b0679d08bfd42804b01b657f663cdb538f41d010459fa92d3363c658381da"}
Jul 07 17:55:10.899437 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:10.898993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:55:10.900522 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:10.900490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mslq2" event={"ID":"0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b","Type":"ContainerStarted","Data":"cfc4aa53575badc3eacea09d1623fa0ff67ae5c96dcd6970c425f1bcf6abf72c"}
Jul 07 17:55:10.917924 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:10.917867    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vmnjw" podStartSLOduration=65.894724932 podStartE2EDuration="1m11.917850274s" totalImagesPullingTime="6.023125342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:10.917102706 +0000 UTC m=+72.807463428" watchObservedRunningTime="2026-07-07 17:55:10.917850274 +0000 UTC m=+72.808210977"
Jul 07 17:55:10.939114 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:10.938731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mslq2" podStartSLOduration=65.927578407 podStartE2EDuration="1m11.938708724s" totalImagesPullingTime="6.011130317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:10.937736843 +0000 UTC m=+72.828097568" watchObservedRunningTime="2026-07-07 17:55:10.938708724 +0000 UTC m=+72.829069450"
Jul 07 17:55:16.278740 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:16.278690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:16.278740 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:16.278745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:16.279193 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:16.278982    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:16.279193 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:16.279065    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls podName:a9f6b888-066a-498a-afab-6da1a3ab8e9c nodeName:}" failed. No retries permitted until 2026-07-07 17:55:32.279043058 +0000 UTC m=+94.169403763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v78vv" (UID: "a9f6b888-066a-498a-afab-6da1a3ab8e9c") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:55:16.279303 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:16.279282    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a9f6b888-066a-498a-afab-6da1a3ab8e9c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:22.874876 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.874760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"]
Jul 07 17:55:22.877169 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.877150    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:22.881516 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.881491    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"]
Jul 07 17:55:22.883502 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.883482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:22.915610 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.915582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 17:55:22.916614 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.916587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"]
Jul 07 17:55:22.916806 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.916790    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 17:55:22.916894 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.916851    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5qxpl\""
Jul 07 17:55:22.917795 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:22.917770    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-f8wcm\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"default-dockercfg-f8wcm\"" type="*v1.Secret"
Jul 07 17:55:22.917926 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:22.917798    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret"
Jul 07 17:55:22.918005 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:22.917933    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap"
Jul 07 17:55:22.918069 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.918010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 17:55:22.918653 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.918639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 17:55:22.924191 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.924168    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"]
Jul 07 17:55:22.928801 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.928772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6da7a0f2-d1cb-4e97-809d-69a52db0d134-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-588496675d-gv5vd\" (UID: \"6da7a0f2-d1cb-4e97-809d-69a52db0d134\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:22.928949 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.928931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:22.929018 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.928971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:22.929082 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:22.929013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxckb\" (UniqueName: \"kubernetes.io/projected/6da7a0f2-d1cb-4e97-809d-69a52db0d134-kube-api-access-fxckb\") pod \"managed-serviceaccount-addon-agent-588496675d-gv5vd\" (UID: \"6da7a0f2-d1cb-4e97-809d-69a52db0d134\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:23.029809 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.029774    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:23.030052 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.029819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:23.030052 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.029850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxckb\" (UniqueName: \"kubernetes.io/projected/6da7a0f2-d1cb-4e97-809d-69a52db0d134-kube-api-access-fxckb\") pod \"managed-serviceaccount-addon-agent-588496675d-gv5vd\" (UID: \"6da7a0f2-d1cb-4e97-809d-69a52db0d134\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:23.030052 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.029885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6da7a0f2-d1cb-4e97-809d-69a52db0d134-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-588496675d-gv5vd\" (UID: \"6da7a0f2-d1cb-4e97-809d-69a52db0d134\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:23.032450 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.032422    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6da7a0f2-d1cb-4e97-809d-69a52db0d134-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-588496675d-gv5vd\" (UID: \"6da7a0f2-d1cb-4e97-809d-69a52db0d134\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:23.063052 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.063015    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxckb\" (UniqueName: \"kubernetes.io/projected/6da7a0f2-d1cb-4e97-809d-69a52db0d134-kube-api-access-fxckb\") pod \"managed-serviceaccount-addon-agent-588496675d-gv5vd\" (UID: \"6da7a0f2-d1cb-4e97-809d-69a52db0d134\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:23.196961 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.196848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"
Jul 07 17:55:23.339690 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.339656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd"]
Jul 07 17:55:23.342855 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:23.342831    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6da7a0f2_d1cb_4e97_809d_69a52db0d134.slice/crio-7b4ed0728b0dfe5ce6cbd035d369d272e14b3629a37f1049870df3278fa3f5f0 WatchSource:0}: Error finding container 7b4ed0728b0dfe5ce6cbd035d369d272e14b3629a37f1049870df3278fa3f5f0: Status 404 returned error can't find the container with id 7b4ed0728b0dfe5ce6cbd035d369d272e14b3629a37f1049870df3278fa3f5f0
Jul 07 17:55:23.762533 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.759664    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-f8wcm\""
Jul 07 17:55:23.774849 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.774821    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 17:55:23.782742 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.782718    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:23.933690 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:23.933647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd" event={"ID":"6da7a0f2-d1cb-4e97-809d-69a52db0d134","Type":"ContainerStarted","Data":"7b4ed0728b0dfe5ce6cbd035d369d272e14b3629a37f1049870df3278fa3f5f0"}
Jul 07 17:55:24.031035 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:24.030953    2580 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: failed to sync configmap cache: timed out waiting for the condition
Jul 07 17:55:24.031035 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:24.031035    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-nginx-conf podName:6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256 nodeName:}" failed. No retries permitted until 2026-07-07 17:55:24.531018624 +0000 UTC m=+86.421379326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-nginx-conf") pod "networking-console-plugin-6bffdf9778-j4l5g" (UID: "6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256") : failed to sync configmap cache: timed out waiting for the condition
Jul 07 17:55:24.479518 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:24.479481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 17:55:24.543838 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:24.543690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:24.544469 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:24.544444    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-j4l5g\" (UID: \"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:24.702984 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:24.702943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"
Jul 07 17:55:24.862275 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:24.862234    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g"]
Jul 07 17:55:24.865876 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:24.865845    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd7a2cd_09e7_4a08_b5fd_ac1b1ca6c256.slice/crio-7cf308b1a2c2aee3424a5fbb1821be6b5664dd724fcc11c600b92e689ebc5ae1 WatchSource:0}: Error finding container 7cf308b1a2c2aee3424a5fbb1821be6b5664dd724fcc11c600b92e689ebc5ae1: Status 404 returned error can't find the container with id 7cf308b1a2c2aee3424a5fbb1821be6b5664dd724fcc11c600b92e689ebc5ae1
Jul 07 17:55:24.937656 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:24.937619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g" event={"ID":"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256","Type":"ContainerStarted","Data":"7cf308b1a2c2aee3424a5fbb1821be6b5664dd724fcc11c600b92e689ebc5ae1"}
Jul 07 17:55:26.948370 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:26.947891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd" event={"ID":"6da7a0f2-d1cb-4e97-809d-69a52db0d134","Type":"ContainerStarted","Data":"a415cdd368fa9a9bcf93f9f19190b2473bbb49c83fa45d94582de28174026307"}
Jul 07 17:55:26.950152 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:26.950127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g" event={"ID":"6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256","Type":"ContainerStarted","Data":"fab01373fe2206a0f836a94a781f2a75dda5683dc51d1355e55dcbea18ac99a8"}
Jul 07 17:55:26.970063 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:26.970005    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-588496675d-gv5vd" podStartSLOduration=2.340704021 podStartE2EDuration="4.969988146s" totalImagesPullingTime="2.629284125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:26.969591288 +0000 UTC m=+88.859952012" watchObservedRunningTime="2026-07-07 17:55:26.969988146 +0000 UTC m=+88.860348870"
Jul 07 17:55:26.990501 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:26.990449    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-j4l5g" podStartSLOduration=3.168240728 podStartE2EDuration="4.99043477s" totalImagesPullingTime="1.822194042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:26.989698762 +0000 UTC m=+88.880059489" watchObservedRunningTime="2026-07-07 17:55:26.99043477 +0000 UTC m=+88.880795494"
Jul 07 17:55:32.303018 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:32.302982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:32.305406 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:32.305381    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a9f6b888-066a-498a-afab-6da1a3ab8e9c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v78vv\" (UID: \"a9f6b888-066a-498a-afab-6da1a3ab8e9c\") " pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:32.548996 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:32.548944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v78vv"
Jul 07 17:55:32.691392 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:32.691359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v78vv"]
Jul 07 17:55:32.701385 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:32.701356    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f6b888_066a_498a_afab_6da1a3ab8e9c.slice/crio-8274ae9d379aca68aa4aada050ea074e27310d4841b9adab9031cf2a1927bfe5 WatchSource:0}: Error finding container 8274ae9d379aca68aa4aada050ea074e27310d4841b9adab9031cf2a1927bfe5: Status 404 returned error can't find the container with id 8274ae9d379aca68aa4aada050ea074e27310d4841b9adab9031cf2a1927bfe5
Jul 07 17:55:32.971884 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:32.971848    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v78vv" event={"ID":"a9f6b888-066a-498a-afab-6da1a3ab8e9c","Type":"ContainerStarted","Data":"8274ae9d379aca68aa4aada050ea074e27310d4841b9adab9031cf2a1927bfe5"}
Jul 07 17:55:35.035417 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.035383    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"]
Jul 07 17:55:35.045474 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.045447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.048572 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.048544    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 17:55:35.048790 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.048769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"]
Jul 07 17:55:35.048853 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.048776    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lbkqr\""
Jul 07 17:55:35.049486 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.049464    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 07 17:55:35.049577 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.049531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 07 17:55:35.049577 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.049563    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 17:55:35.049693 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.049464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 17:55:35.125869 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.125832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.126082 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.125897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.126148 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.126064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq5xw\" (UniqueName: \"kubernetes.io/projected/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-kube-api-access-zq5xw\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.126148 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.126126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.226881 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.226848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.227085 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.226923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.227085 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.226968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq5xw\" (UniqueName: \"kubernetes.io/projected/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-kube-api-access-zq5xw\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.227085 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.227004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.227677 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.227649    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.229897 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.229733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.229897 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.229847    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.238258 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.238234    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq5xw\" (UniqueName: \"kubernetes.io/projected/dc8f062a-97ef-40d9-81f4-9f6b7004d59f-kube-api-access-zq5xw\") pod \"prometheus-operator-567f7b8b7c-88djl\" (UID: \"dc8f062a-97ef-40d9-81f4-9f6b7004d59f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:35.360169 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:35.360124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"
Jul 07 17:55:36.643872 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:36.643841    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-88djl"]
Jul 07 17:55:36.647093 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:36.647052    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8f062a_97ef_40d9_81f4_9f6b7004d59f.slice/crio-80f64218b13d5e579f469baf573658314972c0250771c91bb61c331641ed5c1a WatchSource:0}: Error finding container 80f64218b13d5e579f469baf573658314972c0250771c91bb61c331641ed5c1a: Status 404 returned error can't find the container with id 80f64218b13d5e579f469baf573658314972c0250771c91bb61c331641ed5c1a
Jul 07 17:55:36.988354 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:36.988312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl" event={"ID":"dc8f062a-97ef-40d9-81f4-9f6b7004d59f","Type":"ContainerStarted","Data":"80f64218b13d5e579f469baf573658314972c0250771c91bb61c331641ed5c1a"}
Jul 07 17:55:36.989710 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:36.989687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v78vv" event={"ID":"a9f6b888-066a-498a-afab-6da1a3ab8e9c","Type":"ContainerStarted","Data":"3832771ce3fd427fc2b1c5018467f18ee0b166c64b0f0af8304cbc0e74cec194"}
Jul 07 17:55:37.444567 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.444525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:55:37.444778 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.444580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:55:37.447334 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.447291    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/363e7a50-bf3c-4a89-bb95-8949a1f64c0b-metrics-tls\") pod \"dns-default-662vr\" (UID: \"363e7a50-bf3c-4a89-bb95-8949a1f64c0b\") " pod="openshift-dns/dns-default-662vr"
Jul 07 17:55:37.447490 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.447470    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"image-registry-769d455b8b-nxxxc\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") " pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:55:37.687827 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.687630    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c44kt\""
Jul 07 17:55:37.695272 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.695241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:55:37.709802 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.709594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dtpjm\""
Jul 07 17:55:37.714284 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.714261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-662vr"
Jul 07 17:55:37.863731 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.862752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-769d455b8b-nxxxc"]
Jul 07 17:55:37.881980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.881946    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-662vr"]
Jul 07 17:55:37.995488 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:37.995402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v78vv" event={"ID":"a9f6b888-066a-498a-afab-6da1a3ab8e9c","Type":"ContainerStarted","Data":"110af2c6e5c49aea1708e097b63cf952f0394bbf8a248ef2f471939d1a89be65"}
Jul 07 17:55:38.143351 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:38.143315    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa7404ee_8f82_4389_9523_041ad682e3e2.slice/crio-a1a4e9a39d7dba1893d319061f88c2a8a8e10eb7d9e971ea1ec08738caf3e37e WatchSource:0}: Error finding container a1a4e9a39d7dba1893d319061f88c2a8a8e10eb7d9e971ea1ec08738caf3e37e: Status 404 returned error can't find the container with id a1a4e9a39d7dba1893d319061f88c2a8a8e10eb7d9e971ea1ec08738caf3e37e
Jul 07 17:55:38.144166 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:38.144006    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod363e7a50_bf3c_4a89_bb95_8949a1f64c0b.slice/crio-f5f81dcd410b6324315fc92f9ef514aa732a326b0144bed283a6b354cfdba9bf WatchSource:0}: Error finding container f5f81dcd410b6324315fc92f9ef514aa732a326b0144bed283a6b354cfdba9bf: Status 404 returned error can't find the container with id f5f81dcd410b6324315fc92f9ef514aa732a326b0144bed283a6b354cfdba9bf
Jul 07 17:55:39.001378 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.001331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl" event={"ID":"dc8f062a-97ef-40d9-81f4-9f6b7004d59f","Type":"ContainerStarted","Data":"4113ef5cd59618563b2602e7ccb607470a3d01fd375c93888201645390451328"}
Jul 07 17:55:39.001848 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.001405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl" event={"ID":"dc8f062a-97ef-40d9-81f4-9f6b7004d59f","Type":"ContainerStarted","Data":"8928e40df421c41b9a1f86dadd35f246955a55bff3994efa713e3944f49eb270"}
Jul 07 17:55:39.002770 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.002728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-662vr" event={"ID":"363e7a50-bf3c-4a89-bb95-8949a1f64c0b","Type":"ContainerStarted","Data":"f5f81dcd410b6324315fc92f9ef514aa732a326b0144bed283a6b354cfdba9bf"}
Jul 07 17:55:39.004434 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.004413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc" event={"ID":"fa7404ee-8f82-4389-9523-041ad682e3e2","Type":"ContainerStarted","Data":"5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a"}
Jul 07 17:55:39.004538 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.004454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc" event={"ID":"fa7404ee-8f82-4389-9523-041ad682e3e2","Type":"ContainerStarted","Data":"a1a4e9a39d7dba1893d319061f88c2a8a8e10eb7d9e971ea1ec08738caf3e37e"}
Jul 07 17:55:39.004610 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.004589    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:55:39.055559 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.055500    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc" podStartSLOduration=100.055483433 podStartE2EDuration="1m40.055483433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:39.054426253 +0000 UTC m=+100.944786981" watchObservedRunningTime="2026-07-07 17:55:39.055483433 +0000 UTC m=+100.945844156"
Jul 07 17:55:39.056146 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:39.056103    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-88djl" podStartSLOduration=2.534335886 podStartE2EDuration="4.056090473s" totalImagesPullingTime="1.521754587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:39.023120124 +0000 UTC m=+100.913480848" watchObservedRunningTime="2026-07-07 17:55:39.056090473 +0000 UTC m=+100.946451178"
Jul 07 17:55:41.550630 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.550538    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"]
Jul 07 17:55:41.568531 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.568503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.571355 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.571271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"]
Jul 07 17:55:41.574191 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.574154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 07 17:55:41.574310 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.574160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4ptxg\""
Jul 07 17:55:41.574495 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.574477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 07 17:55:41.652824 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.652793    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gzlfz"]
Jul 07 17:55:41.672134 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.672103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.681300 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.681274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.681425 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.681315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pksmt\" (UniqueName: \"kubernetes.io/projected/fd31438c-a233-4f6d-8b91-999917b80c6f-kube-api-access-pksmt\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.681425 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.681352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd31438c-a233-4f6d-8b91-999917b80c6f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.681425 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.681379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.682110 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.682087    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 17:55:41.682231 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.682213    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 17:55:41.682324 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.682310    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 17:55:41.682401 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.682387    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qjnn2\""
Jul 07 17:55:41.782495 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.782495 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-metrics-client-ca\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.782746 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-wtmp\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.782746 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.782746 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pksmt\" (UniqueName: \"kubernetes.io/projected/fd31438c-a233-4f6d-8b91-999917b80c6f-kube-api-access-pksmt\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.782746 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-sys\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.782746 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.782746 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd31438c-a233-4f6d-8b91-999917b80c6f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:41.782757    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:55:41.782827    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-tls podName:fd31438c-a233-4f6d-8b91-999917b80c6f nodeName:}" failed. No retries permitted until 2026-07-07 17:55:42.282807253 +0000 UTC m=+104.173167970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-2bbsv" (UID: "fd31438c-a233-4f6d-8b91-999917b80c6f") : secret "openshift-state-metrics-tls" not found
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-textfile\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-tls\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.782952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdbcr\" (UniqueName: \"kubernetes.io/projected/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-kube-api-access-tdbcr\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.783045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.783013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-root\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.783457 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.783435    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd31438c-a233-4f6d-8b91-999917b80c6f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.785416 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.785393    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.797962 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.797934    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pksmt\" (UniqueName: \"kubernetes.io/projected/fd31438c-a233-4f6d-8b91-999917b80c6f-kube-api-access-pksmt\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:41.883865 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-sys\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.883865 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-textfile\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-tls\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883960    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-sys\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.883985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdbcr\" (UniqueName: \"kubernetes.io/projected/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-kube-api-access-tdbcr\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-root\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-metrics-client-ca\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884092    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-root\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-wtmp\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884579 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884251    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-wtmp\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884579 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884284    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-textfile\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884709 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884686    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.884762 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.884715    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-metrics-client-ca\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.886565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.886546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.886632 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.886618    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-node-exporter-tls\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.894980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.894955    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdbcr\" (UniqueName: \"kubernetes.io/projected/c7138e6b-f922-40c2-a1e3-2e1fbac844c8-kube-api-access-tdbcr\") pod \"node-exporter-gzlfz\" (UID: \"c7138e6b-f922-40c2-a1e3-2e1fbac844c8\") " pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:41.906728 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.906701    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vmnjw"
Jul 07 17:55:41.981092 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:41.981059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gzlfz"
Jul 07 17:55:42.019704 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:42.019677    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7138e6b_f922_40c2_a1e3_2e1fbac844c8.slice/crio-2173885be5a702f4b2a822dfa0b54827e6cd8970b491be98ab560e8010429783 WatchSource:0}: Error finding container 2173885be5a702f4b2a822dfa0b54827e6cd8970b491be98ab560e8010429783: Status 404 returned error can't find the container with id 2173885be5a702f4b2a822dfa0b54827e6cd8970b491be98ab560e8010429783
Jul 07 17:55:42.021173 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.021129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v78vv" event={"ID":"a9f6b888-066a-498a-afab-6da1a3ab8e9c","Type":"ContainerStarted","Data":"7b24b8ad636a4f489b6197d95fbbb8b1e3cc9ecdc8dffc61d377630537d7a9f6"}
Jul 07 17:55:42.022991 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.022967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-662vr" event={"ID":"363e7a50-bf3c-4a89-bb95-8949a1f64c0b","Type":"ContainerStarted","Data":"94a34222da649783d1901e02887da9c183c82346a721661880c69f675cd1ec58"}
Jul 07 17:55:42.023086 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.023011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-662vr" event={"ID":"363e7a50-bf3c-4a89-bb95-8949a1f64c0b","Type":"ContainerStarted","Data":"d3f77f41fa08affa44f63e9e3650e39eb5e2238347e10b30b604a88dbf6ce2b3"}
Jul 07 17:55:42.023180 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.023165    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-662vr"
Jul 07 17:55:42.048040 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.047991    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v78vv" podStartSLOduration=33.852967486 podStartE2EDuration="42.047972939s" totalImagesPullingTime="8.195005453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:55:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:42.046923342 +0000 UTC m=+103.937284058" watchObservedRunningTime="2026-07-07 17:55:42.047972939 +0000 UTC m=+103.938333641"
Jul 07 17:55:42.070620 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.070567    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-662vr" podStartSLOduration=66.184728056 podStartE2EDuration="1m9.070550616s" totalImagesPullingTime="2.88582256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:42.069648897 +0000 UTC m=+103.960009612" watchObservedRunningTime="2026-07-07 17:55:42.070550616 +0000 UTC m=+103.960911340"
Jul 07 17:55:42.288606 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.288525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:42.291155 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.291132    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fd31438c-a233-4f6d-8b91-999917b80c6f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-2bbsv\" (UID: \"fd31438c-a233-4f6d-8b91-999917b80c6f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:42.479715 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.479675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"
Jul 07 17:55:42.684405 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:42.684367    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv"]
Jul 07 17:55:42.688120 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:42.688097    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd31438c_a233_4f6d_8b91_999917b80c6f.slice/crio-7b132215d055f6f87721cc3fcc93b62b2909ac8efb0d339009814fcd297797da WatchSource:0}: Error finding container 7b132215d055f6f87721cc3fcc93b62b2909ac8efb0d339009814fcd297797da: Status 404 returned error can't find the container with id 7b132215d055f6f87721cc3fcc93b62b2909ac8efb0d339009814fcd297797da
Jul 07 17:55:43.028442 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:43.028374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzlfz" event={"ID":"c7138e6b-f922-40c2-a1e3-2e1fbac844c8","Type":"ContainerStarted","Data":"2173885be5a702f4b2a822dfa0b54827e6cd8970b491be98ab560e8010429783"}
Jul 07 17:55:43.030430 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:43.030387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv" event={"ID":"fd31438c-a233-4f6d-8b91-999917b80c6f","Type":"ContainerStarted","Data":"0158990fb3106c32abeedc7df856835f89ec825a4e48eddafb9498c95f897f38"}
Jul 07 17:55:43.030538 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:43.030451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv" event={"ID":"fd31438c-a233-4f6d-8b91-999917b80c6f","Type":"ContainerStarted","Data":"e29f175eaa1495faec35aa3731bda12cd9780658390c50d5c61161158e329996"}
Jul 07 17:55:43.030538 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:43.030480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv" event={"ID":"fd31438c-a233-4f6d-8b91-999917b80c6f","Type":"ContainerStarted","Data":"7b132215d055f6f87721cc3fcc93b62b2909ac8efb0d339009814fcd297797da"}
Jul 07 17:55:44.035782 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:44.035749    2580 generic.go:356] "Generic (PLEG): container finished" podID="c7138e6b-f922-40c2-a1e3-2e1fbac844c8" containerID="75a38f45ad8fc6d22f80583bb963f12c60b8134914454c9dcb86f203d4a24bcf" exitCode=0
Jul 07 17:55:44.036219 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:44.035833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzlfz" event={"ID":"c7138e6b-f922-40c2-a1e3-2e1fbac844c8","Type":"ContainerDied","Data":"75a38f45ad8fc6d22f80583bb963f12c60b8134914454c9dcb86f203d4a24bcf"}
Jul 07 17:55:45.042037 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:45.042003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzlfz" event={"ID":"c7138e6b-f922-40c2-a1e3-2e1fbac844c8","Type":"ContainerStarted","Data":"af92f54c6e69f68abf6ae4795c8f98af413aed66ee993d393f3ed3069ce5b352"}
Jul 07 17:55:45.042487 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:45.042055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzlfz" event={"ID":"c7138e6b-f922-40c2-a1e3-2e1fbac844c8","Type":"ContainerStarted","Data":"605bc38849c1e16b68fcf3b4f5dc243618f205b85e6419d541e22f1a2f6c5321"}
Jul 07 17:55:45.044068 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:45.044043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv" event={"ID":"fd31438c-a233-4f6d-8b91-999917b80c6f","Type":"ContainerStarted","Data":"ebadc669e60c33cc6d6b584e4923cd1527fba64d2433637c70d6fd5682935a06"}
Jul 07 17:55:45.067603 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:45.067552    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gzlfz" podStartSLOduration=2.548381211 podStartE2EDuration="4.067536081s" totalImagesPullingTime="1.402083414s" totalInitContainerRuntime="117.071456ms" isStatefulPod=true podCreationTimestamp="2026-07-07 17:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:45.066676748 +0000 UTC m=+106.957037483" watchObservedRunningTime="2026-07-07 17:55:45.067536081 +0000 UTC m=+106.957896801"
Jul 07 17:55:45.090526 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:45.090481    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-2bbsv" podStartSLOduration=2.764301725 podStartE2EDuration="4.090465332s" totalImagesPullingTime="1.326163607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:55:45.089613025 +0000 UTC m=+106.979973748" watchObservedRunningTime="2026-07-07 17:55:45.090465332 +0000 UTC m=+106.980826061"
Jul 07 17:55:45.955734 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:45.955700    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-769d455b8b-nxxxc"]
Jul 07 17:55:49.782337 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.782302    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-fwc45"]
Jul 07 17:55:49.784323 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.784308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:55:49.789032 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.789010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 17:55:49.789283 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.789262    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pkdbm\""
Jul 07 17:55:49.789405 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.789288    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 17:55:49.809640 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.809601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-fwc45"]
Jul 07 17:55:49.949617 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:49.949568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvn2l\" (UniqueName: \"kubernetes.io/projected/4bdb156a-5c6c-4901-a877-045041cf910d-kube-api-access-kvn2l\") pod \"downloads-685bfb5f64-fwc45\" (UID: \"4bdb156a-5c6c-4901-a877-045041cf910d\") " pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:55:50.050457 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:50.050388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvn2l\" (UniqueName: \"kubernetes.io/projected/4bdb156a-5c6c-4901-a877-045041cf910d-kube-api-access-kvn2l\") pod \"downloads-685bfb5f64-fwc45\" (UID: \"4bdb156a-5c6c-4901-a877-045041cf910d\") " pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:55:50.066730 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:50.066702    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvn2l\" (UniqueName: \"kubernetes.io/projected/4bdb156a-5c6c-4901-a877-045041cf910d-kube-api-access-kvn2l\") pod \"downloads-685bfb5f64-fwc45\" (UID: \"4bdb156a-5c6c-4901-a877-045041cf910d\") " pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:55:50.092593 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:50.092562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:55:50.225431 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:50.225400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-fwc45"]
Jul 07 17:55:50.229139 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:55:50.229095    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdb156a_5c6c_4901_a877_045041cf910d.slice/crio-77b49efe52c4c453e44a417ae32d5f43425a2e1ab73ec04e09c89d981e999374 WatchSource:0}: Error finding container 77b49efe52c4c453e44a417ae32d5f43425a2e1ab73ec04e09c89d981e999374: Status 404 returned error can't find the container with id 77b49efe52c4c453e44a417ae32d5f43425a2e1ab73ec04e09c89d981e999374
Jul 07 17:55:51.071379 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:51.071343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-fwc45" event={"ID":"4bdb156a-5c6c-4901-a877-045041cf910d","Type":"ContainerStarted","Data":"77b49efe52c4c453e44a417ae32d5f43425a2e1ab73ec04e09c89d981e999374"}
Jul 07 17:55:52.033365 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:52.033335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-662vr"
Jul 07 17:55:55.962333 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:55.962305    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:55:59.703118 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.703083    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cdc9db4b5-cxzlg"]
Jul 07 17:55:59.708057 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.708034    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.714065 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.714036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 17:55:59.714206 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.714171    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 17:55:59.714206 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.714036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 17:55:59.714335 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.714043    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 17:55:59.714980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.714861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 17:55:59.714980 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.714869    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c8lhm\""
Jul 07 17:55:59.725001 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.724978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdc9db4b5-cxzlg"]
Jul 07 17:55:59.836539 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.836502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-service-ca\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.836727 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.836558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-oauth-config\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.836727 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.836630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbsn7\" (UniqueName: \"kubernetes.io/projected/8276e503-386c-4614-99b5-f39d5d89de74-kube-api-access-xbsn7\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.836727 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.836679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-oauth-serving-cert\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.836727 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.836716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-serving-cert\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.836936 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.836791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-console-config\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.937154 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-service-ca\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.937154 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-oauth-config\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.937426 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbsn7\" (UniqueName: \"kubernetes.io/projected/8276e503-386c-4614-99b5-f39d5d89de74-kube-api-access-xbsn7\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.937426 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-oauth-serving-cert\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.937504 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-serving-cert\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.937504 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-console-config\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.938181 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.937984    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-service-ca\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.938181 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.938141    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-console-config\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.938370 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.938217    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-oauth-serving-cert\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.940160 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.940140    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-oauth-config\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.940445 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.940421    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-serving-cert\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:55:59.948218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:55:59.948192    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbsn7\" (UniqueName: \"kubernetes.io/projected/8276e503-386c-4614-99b5-f39d5d89de74-kube-api-access-xbsn7\") pod \"console-cdc9db4b5-cxzlg\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") " pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:00.021329 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:00.021230    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:00.194208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:00.194184    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdc9db4b5-cxzlg"]
Jul 07 17:56:00.196433 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:56:00.196397    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8276e503_386c_4614_99b5_f39d5d89de74.slice/crio-2f0c47d3b666022fab811406dd16ae2c58ae0ec41b3c9081f861af1919185a49 WatchSource:0}: Error finding container 2f0c47d3b666022fab811406dd16ae2c58ae0ec41b3c9081f861af1919185a49: Status 404 returned error can't find the container with id 2f0c47d3b666022fab811406dd16ae2c58ae0ec41b3c9081f861af1919185a49
Jul 07 17:56:01.118887 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:01.118842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc9db4b5-cxzlg" event={"ID":"8276e503-386c-4614-99b5-f39d5d89de74","Type":"ContainerStarted","Data":"2f0c47d3b666022fab811406dd16ae2c58ae0ec41b3c9081f861af1919185a49"}
Jul 07 17:56:02.002543 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.002508    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5988b86fc-swgx5"]
Jul 07 17:56:02.005289 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.005259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.016674 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.016644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 17:56:02.018515 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.018490    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5988b86fc-swgx5"]
Jul 07 17:56:02.052330 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-config\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.052507 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-oauth-config\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.052507 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-service-ca\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.052507 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-serving-cert\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.052507 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-trusted-ca-bundle\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.052507 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8gfk\" (UniqueName: \"kubernetes.io/projected/66a4ee05-3a27-42fd-880a-b6104d1a3b99-kube-api-access-m8gfk\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.052717 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.052521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-oauth-serving-cert\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153041 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-serving-cert\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153520 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-trusted-ca-bundle\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153520 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8gfk\" (UniqueName: \"kubernetes.io/projected/66a4ee05-3a27-42fd-880a-b6104d1a3b99-kube-api-access-m8gfk\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153520 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-oauth-serving-cert\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153520 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-config\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153520 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-oauth-config\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.153520 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.153271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-service-ca\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.154224 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.154197    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-oauth-serving-cert\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.154324 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.154248    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-service-ca\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.154459 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.154437    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-config\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.154505 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.154482    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-trusted-ca-bundle\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.156045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.156022    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-oauth-config\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.156252 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.156229    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-serving-cert\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.167337 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.167299    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8gfk\" (UniqueName: \"kubernetes.io/projected/66a4ee05-3a27-42fd-880a-b6104d1a3b99-kube-api-access-m8gfk\") pod \"console-5988b86fc-swgx5\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") " pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:02.317342 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:02.317246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:07.579740 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:07.579698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5988b86fc-swgx5"]
Jul 07 17:56:07.583184 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:56:07.583155    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a4ee05_3a27_42fd_880a_b6104d1a3b99.slice/crio-493d52701961d3e29745dee2e460d9e52be2cef52596501062d9e52fc054b0dd WatchSource:0}: Error finding container 493d52701961d3e29745dee2e460d9e52be2cef52596501062d9e52fc054b0dd: Status 404 returned error can't find the container with id 493d52701961d3e29745dee2e460d9e52be2cef52596501062d9e52fc054b0dd
Jul 07 17:56:08.151628 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.151549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-fwc45" event={"ID":"4bdb156a-5c6c-4901-a877-045041cf910d","Type":"ContainerStarted","Data":"eac23d83b2373de643f26cfffd16c4652c9149a3d5458ad95928dd9c9179e2a5"}
Jul 07 17:56:08.152565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.152515    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:56:08.154269 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.154235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5988b86fc-swgx5" event={"ID":"66a4ee05-3a27-42fd-880a-b6104d1a3b99","Type":"ContainerStarted","Data":"493d52701961d3e29745dee2e460d9e52be2cef52596501062d9e52fc054b0dd"}
Jul 07 17:56:08.169261 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.169193    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-fwc45"
Jul 07 17:56:08.183234 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.183040    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-fwc45" podStartSLOduration=1.845290997 podStartE2EDuration="19.183018129s" totalImagesPullingTime="17.337727132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:56:08.181970515 +0000 UTC m=+130.072331240" watchObservedRunningTime="2026-07-07 17:56:08.183018129 +0000 UTC m=+130.073378856"
Jul 07 17:56:08.399148 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.399102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:56:08.403367 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.403208    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1bba2b-09b2-4a4f-ab55-506d6180ecf8-metrics-certs\") pod \"network-metrics-daemon-8gfg2\" (UID: \"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8\") " pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:56:08.417178 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.417139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g4wsg\""
Jul 07 17:56:08.425435 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.425189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8gfg2"
Jul 07 17:56:08.632454 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:08.632417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8gfg2"]
Jul 07 17:56:08.636727 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:56:08.636565    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1bba2b_09b2_4a4f_ab55_506d6180ecf8.slice/crio-93c2937ea369ae086322948034636757ad3149f49d2a88b3ffb58c9e0cb7cee8 WatchSource:0}: Error finding container 93c2937ea369ae086322948034636757ad3149f49d2a88b3ffb58c9e0cb7cee8: Status 404 returned error can't find the container with id 93c2937ea369ae086322948034636757ad3149f49d2a88b3ffb58c9e0cb7cee8
Jul 07 17:56:09.163256 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:09.163195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8gfg2" event={"ID":"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8","Type":"ContainerStarted","Data":"93c2937ea369ae086322948034636757ad3149f49d2a88b3ffb58c9e0cb7cee8"}
Jul 07 17:56:10.976145 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:10.976066    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc" podUID="fa7404ee-8f82-4389-9523-041ad682e3e2" containerName="registry" containerID="cri-o://5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a" gracePeriod=30
Jul 07 17:56:12.110170 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.110135    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-bound-sa-token\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138473    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138504    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-certificates\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-trusted-ca\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-installation-pull-secrets\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138612    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-image-registry-private-configuration\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lc8p\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-kube-api-access-4lc8p\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.138929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.138678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa7404ee-8f82-4389-9523-041ad682e3e2-ca-trust-extracted\") pod \"fa7404ee-8f82-4389-9523-041ad682e3e2\" (UID: \"fa7404ee-8f82-4389-9523-041ad682e3e2\") "
Jul 07 17:56:12.140301 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.140192    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:56:12.141685 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.140965    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:56:12.150929 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.150855    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-kube-api-access-4lc8p" (OuterVolumeSpecName: "kube-api-access-4lc8p") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "kube-api-access-4lc8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:56:12.153824 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.153792    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:56:12.154951 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.154870    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:56:12.155045 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.154969    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:56:12.156077 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.156052    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa7404ee-8f82-4389-9523-041ad682e3e2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:56:12.157409 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.157274    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fa7404ee-8f82-4389-9523-041ad682e3e2" (UID: "fa7404ee-8f82-4389-9523-041ad682e3e2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:56:12.181057 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.181022    2580 generic.go:356] "Generic (PLEG): container finished" podID="fa7404ee-8f82-4389-9523-041ad682e3e2" containerID="5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a" exitCode=0
Jul 07 17:56:12.181213 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.181117    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc"
Jul 07 17:56:12.181441 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.181134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc" event={"ID":"fa7404ee-8f82-4389-9523-041ad682e3e2","Type":"ContainerDied","Data":"5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a"}
Jul 07 17:56:12.181529 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.181479    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769d455b8b-nxxxc" event={"ID":"fa7404ee-8f82-4389-9523-041ad682e3e2","Type":"ContainerDied","Data":"a1a4e9a39d7dba1893d319061f88c2a8a8e10eb7d9e971ea1ec08738caf3e37e"}
Jul 07 17:56:12.181529 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.181512    2580 scope.go:117] "RemoveContainer" containerID="5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a"
Jul 07 17:56:12.205066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.204949    2580 scope.go:117] "RemoveContainer" containerID="5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a"
Jul 07 17:56:12.208379 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:56:12.207845    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a\": container with ID starting with 5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a not found: ID does not exist" containerID="5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a"
Jul 07 17:56:12.208379 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.207891    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a"} err="failed to get container status \"5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a\": rpc error: code = NotFound desc = could not find container \"5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a\": container with ID starting with 5915190c08b66c983a453f0ce69476fa9bab2b6bc464d37716d66972485cbe8a not found: ID does not exist"
Jul 07 17:56:12.225441 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.225408    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-769d455b8b-nxxxc"]
Jul 07 17:56:12.231075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.231046    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-769d455b8b-nxxxc"]
Jul 07 17:56:12.239497 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239470    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-bound-sa-token\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239497 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239496    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-tls\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239506    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-registry-certificates\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239520    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa7404ee-8f82-4389-9523-041ad682e3e2-trusted-ca\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239533    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-installation-pull-secrets\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239543    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa7404ee-8f82-4389-9523-041ad682e3e2-image-registry-private-configuration\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239552    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lc8p\" (UniqueName: \"kubernetes.io/projected/fa7404ee-8f82-4389-9523-041ad682e3e2-kube-api-access-4lc8p\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.239679 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.239565    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa7404ee-8f82-4389-9523-041ad682e3e2-ca-trust-extracted\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:12.595149 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:12.595118    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa7404ee-8f82-4389-9523-041ad682e3e2" path="/var/lib/kubelet/pods/fa7404ee-8f82-4389-9523-041ad682e3e2/volumes"
Jul 07 17:56:13.187729 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.187645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8gfg2" event={"ID":"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8","Type":"ContainerStarted","Data":"819b62a7b269ad1a9a5e5a456eb8ae7aca6bae81cdbe5d1cf999e8a83394c193"}
Jul 07 17:56:13.187729 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.187710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8gfg2" event={"ID":"5f1bba2b-09b2-4a4f-ab55-506d6180ecf8","Type":"ContainerStarted","Data":"dcd12849b65af455da7feee1a65fae9526571c067c43f73e2c1e93d158634067"}
Jul 07 17:56:13.190362 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.190331    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5988b86fc-swgx5" event={"ID":"66a4ee05-3a27-42fd-880a-b6104d1a3b99","Type":"ContainerStarted","Data":"9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158"}
Jul 07 17:56:13.192008 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.191987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc9db4b5-cxzlg" event={"ID":"8276e503-386c-4614-99b5-f39d5d89de74","Type":"ContainerStarted","Data":"46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7"}
Jul 07 17:56:13.209513 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.209443    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8gfg2" podStartSLOduration=130.774308677 podStartE2EDuration="2m14.209421887s" totalImagesPullingTime="3.43511321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:56:13.207747183 +0000 UTC m=+135.098107908" watchObservedRunningTime="2026-07-07 17:56:13.209421887 +0000 UTC m=+135.099782614"
Jul 07 17:56:13.226624 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.226563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cdc9db4b5-cxzlg" podStartSLOduration=2.350148537 podStartE2EDuration="14.226546151s" totalImagesPullingTime="11.876397614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:56:13.225598473 +0000 UTC m=+135.115959207" watchObservedRunningTime="2026-07-07 17:56:13.226546151 +0000 UTC m=+135.116906875"
Jul 07 17:56:13.245598 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:13.245542    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5988b86fc-swgx5" podStartSLOduration=7.747315893 podStartE2EDuration="12.245526258s" totalImagesPullingTime="4.498210365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:56:13.243609884 +0000 UTC m=+135.133970607" watchObservedRunningTime="2026-07-07 17:56:13.245526258 +0000 UTC m=+135.135886982"
Jul 07 17:56:20.022083 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:20.022042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:20.022533 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:20.022113    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:20.026992 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:20.026971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:20.224531 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:20.224501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:22.317630 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:22.317590    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:22.318092 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:22.317680    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:22.322410 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:22.322386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:23.234719 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:23.234692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:56:23.293642 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:23.293612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdc9db4b5-cxzlg"]
Jul 07 17:56:48.315237 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.315193    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-cdc9db4b5-cxzlg" podUID="8276e503-386c-4614-99b5-f39d5d89de74" containerName="console" containerID="cri-o://46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7" gracePeriod=15
Jul 07 17:56:48.624480 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.624446    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:48.732175 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732138    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-service-ca\") pod \"8276e503-386c-4614-99b5-f39d5d89de74\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") "
Jul 07 17:56:48.732175 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-oauth-serving-cert\") pod \"8276e503-386c-4614-99b5-f39d5d89de74\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") "
Jul 07 17:56:48.732403 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732217    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-oauth-config\") pod \"8276e503-386c-4614-99b5-f39d5d89de74\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") "
Jul 07 17:56:48.732403 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbsn7\" (UniqueName: \"kubernetes.io/projected/8276e503-386c-4614-99b5-f39d5d89de74-kube-api-access-xbsn7\") pod \"8276e503-386c-4614-99b5-f39d5d89de74\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") "
Jul 07 17:56:48.732403 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732273    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-serving-cert\") pod \"8276e503-386c-4614-99b5-f39d5d89de74\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") "
Jul 07 17:56:48.732403 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732294    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-console-config\") pod \"8276e503-386c-4614-99b5-f39d5d89de74\" (UID: \"8276e503-386c-4614-99b5-f39d5d89de74\") "
Jul 07 17:56:48.732792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732648    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8276e503-386c-4614-99b5-f39d5d89de74" (UID: "8276e503-386c-4614-99b5-f39d5d89de74"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:56:48.732792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732674    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-service-ca" (OuterVolumeSpecName: "service-ca") pod "8276e503-386c-4614-99b5-f39d5d89de74" (UID: "8276e503-386c-4614-99b5-f39d5d89de74"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:56:48.732792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.732758    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-console-config" (OuterVolumeSpecName: "console-config") pod "8276e503-386c-4614-99b5-f39d5d89de74" (UID: "8276e503-386c-4614-99b5-f39d5d89de74"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:56:48.734717 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.734686    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8276e503-386c-4614-99b5-f39d5d89de74" (UID: "8276e503-386c-4614-99b5-f39d5d89de74"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:56:48.734800 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.734726    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8276e503-386c-4614-99b5-f39d5d89de74" (UID: "8276e503-386c-4614-99b5-f39d5d89de74"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:56:48.734800 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.734751    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8276e503-386c-4614-99b5-f39d5d89de74-kube-api-access-xbsn7" (OuterVolumeSpecName: "kube-api-access-xbsn7") pod "8276e503-386c-4614-99b5-f39d5d89de74" (UID: "8276e503-386c-4614-99b5-f39d5d89de74"). InnerVolumeSpecName "kube-api-access-xbsn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:56:48.833391 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.833349    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-oauth-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:48.833391 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.833381    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-oauth-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:48.833391 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.833390    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbsn7\" (UniqueName: \"kubernetes.io/projected/8276e503-386c-4614-99b5-f39d5d89de74-kube-api-access-xbsn7\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:48.833391 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.833399    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8276e503-386c-4614-99b5-f39d5d89de74-console-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:48.833637 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.833407    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-console-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:48.833637 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:48.833416    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8276e503-386c-4614-99b5-f39d5d89de74-service-ca\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:56:49.328330 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.328293    2580 generic.go:356] "Generic (PLEG): container finished" podID="8276e503-386c-4614-99b5-f39d5d89de74" containerID="46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7" exitCode=0
Jul 07 17:56:49.328749 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.328358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc9db4b5-cxzlg" event={"ID":"8276e503-386c-4614-99b5-f39d5d89de74","Type":"ContainerDied","Data":"46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7"}
Jul 07 17:56:49.328749 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.328375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdc9db4b5-cxzlg"
Jul 07 17:56:49.328749 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.328435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdc9db4b5-cxzlg" event={"ID":"8276e503-386c-4614-99b5-f39d5d89de74","Type":"ContainerDied","Data":"2f0c47d3b666022fab811406dd16ae2c58ae0ec41b3c9081f861af1919185a49"}
Jul 07 17:56:49.328749 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.328462    2580 scope.go:117] "RemoveContainer" containerID="46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7"
Jul 07 17:56:49.337837 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.337819    2580 scope.go:117] "RemoveContainer" containerID="46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7"
Jul 07 17:56:49.338167 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:56:49.338142    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7\": container with ID starting with 46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7 not found: ID does not exist" containerID="46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7"
Jul 07 17:56:49.338226 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.338176    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7"} err="failed to get container status \"46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7\": rpc error: code = NotFound desc = could not find container \"46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7\": container with ID starting with 46a12711c469c013e3b9413be15e999385d7cc71a6ef283f32f908f83ec2d2c7 not found: ID does not exist"
Jul 07 17:56:49.355381 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.355347    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdc9db4b5-cxzlg"]
Jul 07 17:56:49.359155 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:49.359128    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cdc9db4b5-cxzlg"]
Jul 07 17:56:50.592930 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:56:50.592882    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8276e503-386c-4614-99b5-f39d5d89de74" path="/var/lib/kubelet/pods/8276e503-386c-4614-99b5-f39d5d89de74/volumes"
Jul 07 17:57:06.092766 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.092727    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-89d6cf79-6md5j"]
Jul 07 17:57:06.093211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.093015    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa7404ee-8f82-4389-9523-041ad682e3e2" containerName="registry"
Jul 07 17:57:06.093211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.093032    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa7404ee-8f82-4389-9523-041ad682e3e2" containerName="registry"
Jul 07 17:57:06.093211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.093048    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8276e503-386c-4614-99b5-f39d5d89de74" containerName="console"
Jul 07 17:57:06.093211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.093053    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8276e503-386c-4614-99b5-f39d5d89de74" containerName="console"
Jul 07 17:57:06.093211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.093119    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa7404ee-8f82-4389-9523-041ad682e3e2" containerName="registry"
Jul 07 17:57:06.093211 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.093128    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8276e503-386c-4614-99b5-f39d5d89de74" containerName="console"
Jul 07 17:57:06.096363 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.096337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.099633 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.099505    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 07 17:57:06.100101 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.100078    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 07 17:57:06.100533 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.100190    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 07 17:57:06.103027 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.100721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 07 17:57:06.103027 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.101444    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5vhzk\""
Jul 07 17:57:06.103027 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.101883    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 07 17:57:06.108949 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.108241    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 07 17:57:06.110441 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.110416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-89d6cf79-6md5j"]
Jul 07 17:57:06.258022 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.257981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.258362 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.258340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-federate-client-tls\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.258507 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.258493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-metrics-client-ca\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.258631 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.258618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntgmn\" (UniqueName: \"kubernetes.io/projected/9655d5c2-690a-4e16-b9af-83a223e23aa3-kube-api-access-ntgmn\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.258763 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.258750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-secret-telemeter-client\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.258898 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.258885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-serving-certs-ca-bundle\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.259092 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.259077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.259214 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.259200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-telemeter-client-tls\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.359964 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.359891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-telemeter-client-tls\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360163 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.359989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360163 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-federate-client-tls\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360163 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-metrics-client-ca\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360163 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntgmn\" (UniqueName: \"kubernetes.io/projected/9655d5c2-690a-4e16-b9af-83a223e23aa3-kube-api-access-ntgmn\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360163 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-secret-telemeter-client\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360400 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-serving-certs-ca-bundle\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360400 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360967 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360868    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-metrics-client-ca\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.360967 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.360967    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.361594 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.361566    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9655d5c2-690a-4e16-b9af-83a223e23aa3-serving-certs-ca-bundle\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.363019 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.362990    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.363164 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.363147    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-federate-client-tls\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.363218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.363158    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-secret-telemeter-client\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.363218 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.363186    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9655d5c2-690a-4e16-b9af-83a223e23aa3-telemeter-client-tls\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.369277 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.369247    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntgmn\" (UniqueName: \"kubernetes.io/projected/9655d5c2-690a-4e16-b9af-83a223e23aa3-kube-api-access-ntgmn\") pod \"telemeter-client-89d6cf79-6md5j\" (UID: \"9655d5c2-690a-4e16-b9af-83a223e23aa3\") " pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.412414 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.412374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j"
Jul 07 17:57:06.576309 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:06.576285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-89d6cf79-6md5j"]
Jul 07 17:57:06.578933 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:57:06.578884    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9655d5c2_690a_4e16_b9af_83a223e23aa3.slice/crio-b36fb095084efed1b5d4cdd3fd46638ce87934b346ce88f11c387a46df02ad28 WatchSource:0}: Error finding container b36fb095084efed1b5d4cdd3fd46638ce87934b346ce88f11c387a46df02ad28: Status 404 returned error can't find the container with id b36fb095084efed1b5d4cdd3fd46638ce87934b346ce88f11c387a46df02ad28
Jul 07 17:57:07.395009 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:07.394967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j" event={"ID":"9655d5c2-690a-4e16-b9af-83a223e23aa3","Type":"ContainerStarted","Data":"b36fb095084efed1b5d4cdd3fd46638ce87934b346ce88f11c387a46df02ad28"}
Jul 07 17:57:09.405021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:09.404981    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j" event={"ID":"9655d5c2-690a-4e16-b9af-83a223e23aa3","Type":"ContainerStarted","Data":"69df2ce772ca4affea311dcfcb454515546f45534f31a4d60945ae5233ca3056"}
Jul 07 17:57:10.411066 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:10.411029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j" event={"ID":"9655d5c2-690a-4e16-b9af-83a223e23aa3","Type":"ContainerStarted","Data":"80e3681a051319985b260889e6135d4e219817e869f70f5339b9c71f32da284f"}
Jul 07 17:57:10.411464 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:10.411085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j" event={"ID":"9655d5c2-690a-4e16-b9af-83a223e23aa3","Type":"ContainerStarted","Data":"f68bb1630f2ecb887aee4ddf883a196b7e9d90898d5cb556421aea0e24a33c3c"}
Jul 07 17:57:10.440715 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:10.440662    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-89d6cf79-6md5j" podStartSLOduration=1.2748394 podStartE2EDuration="4.440645249s" totalImagesPullingTime="3.165805849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:57:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:57:10.43993966 +0000 UTC m=+192.330300381" watchObservedRunningTime="2026-07-07 17:57:10.440645249 +0000 UTC m=+192.331005973"
Jul 07 17:57:11.410195 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.410158    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68f479448f-dvnxj"]
Jul 07 17:57:11.412529 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.412501    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.427809 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.427779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f479448f-dvnxj"]
Jul 07 17:57:11.500706 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.500663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-serving-cert\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.500706 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.500710    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-service-ca\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.500990 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.500735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-oauth-serving-cert\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.500990 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.500776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-trusted-ca-bundle\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.500990 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.500930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-oauth-config\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.500990 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.500980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525rs\" (UniqueName: \"kubernetes.io/projected/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-kube-api-access-525rs\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.501206 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.501151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-config\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-config\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-serving-cert\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602704 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-service-ca\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602704 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-oauth-serving-cert\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602704 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-trusted-ca-bundle\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602704 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-oauth-config\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.602704 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.602583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525rs\" (UniqueName: \"kubernetes.io/projected/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-kube-api-access-525rs\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.603701 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.603671    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-service-ca\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.603859 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.603831    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-config\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.603957 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.603735    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-oauth-serving-cert\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.604142 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.604116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-trusted-ca-bundle\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.609560 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.609532    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-oauth-config\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.609669 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.609571    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-serving-cert\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.613529 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.613503    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-525rs\" (UniqueName: \"kubernetes.io/projected/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-kube-api-access-525rs\") pod \"console-68f479448f-dvnxj\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") " pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.722410 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.722314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:11.860077 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:11.860043    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f479448f-dvnxj"]
Jul 07 17:57:11.863294 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:57:11.863268    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfd7d9d_2e38_4c71_b503_d861c1891ee9.slice/crio-cb81b1f0d037c4e740cf6e9d14071bdaa92364475b6aebfc5607f6617eeaf350 WatchSource:0}: Error finding container cb81b1f0d037c4e740cf6e9d14071bdaa92364475b6aebfc5607f6617eeaf350: Status 404 returned error can't find the container with id cb81b1f0d037c4e740cf6e9d14071bdaa92364475b6aebfc5607f6617eeaf350
Jul 07 17:57:12.422048 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:12.422012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f479448f-dvnxj" event={"ID":"0cfd7d9d-2e38-4c71-b503-d861c1891ee9","Type":"ContainerStarted","Data":"ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f"}
Jul 07 17:57:12.422468 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:12.422072    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f479448f-dvnxj" event={"ID":"0cfd7d9d-2e38-4c71-b503-d861c1891ee9","Type":"ContainerStarted","Data":"cb81b1f0d037c4e740cf6e9d14071bdaa92364475b6aebfc5607f6617eeaf350"}
Jul 07 17:57:12.442772 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:12.442714    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68f479448f-dvnxj" podStartSLOduration=1.4426987709999999 podStartE2EDuration="1.442698771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:57:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:57:12.441421067 +0000 UTC m=+194.331781793" watchObservedRunningTime="2026-07-07 17:57:12.442698771 +0000 UTC m=+194.333059496"
Jul 07 17:57:21.723077 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:21.723023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:21.723527 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:21.723105    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:21.727992 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:21.727965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:22.465282 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:22.465240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:57:22.522866 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:22.522823    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5988b86fc-swgx5"]
Jul 07 17:57:47.551393 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.551347    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5988b86fc-swgx5" podUID="66a4ee05-3a27-42fd-880a-b6104d1a3b99" containerName="console" containerID="cri-o://9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158" gracePeriod=15
Jul 07 17:57:47.810725 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.810657    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:57:47.883017 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.882976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-oauth-config\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-oauth-serving-cert\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883061    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8gfk\" (UniqueName: \"kubernetes.io/projected/66a4ee05-3a27-42fd-880a-b6104d1a3b99-kube-api-access-m8gfk\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883141    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-config\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883208 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883185    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-trusted-ca-bundle\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883412 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883258    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-service-ca\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883412 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883284    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-serving-cert\") pod \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\" (UID: \"66a4ee05-3a27-42fd-880a-b6104d1a3b99\") "
Jul 07 17:57:47.883531 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883413    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:57:47.883638 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883609    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-config" (OuterVolumeSpecName: "console-config") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:57:47.883762 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.883611    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:57:47.884075 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.884048    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-service-ca" (OuterVolumeSpecName: "service-ca") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:57:47.885451 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.885419    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:57:47.885451 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.885438    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:57:47.885584 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.885479    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a4ee05-3a27-42fd-880a-b6104d1a3b99-kube-api-access-m8gfk" (OuterVolumeSpecName: "kube-api-access-m8gfk") pod "66a4ee05-3a27-42fd-880a-b6104d1a3b99" (UID: "66a4ee05-3a27-42fd-880a-b6104d1a3b99"). InnerVolumeSpecName "kube-api-access-m8gfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:57:47.984061 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984009    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:47.984061 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984054    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-trusted-ca-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:47.984061 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984063    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-service-ca\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:47.984061 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984072    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:47.984061 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984081    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66a4ee05-3a27-42fd-880a-b6104d1a3b99-console-oauth-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:47.984357 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984089    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66a4ee05-3a27-42fd-880a-b6104d1a3b99-oauth-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:47.984357 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:47.984099    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8gfk\" (UniqueName: \"kubernetes.io/projected/66a4ee05-3a27-42fd-880a-b6104d1a3b99-kube-api-access-m8gfk\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:57:48.555729 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.555684    2580 generic.go:356] "Generic (PLEG): container finished" podID="66a4ee05-3a27-42fd-880a-b6104d1a3b99" containerID="9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158" exitCode=0
Jul 07 17:57:48.555729 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.555733    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5988b86fc-swgx5"
Jul 07 17:57:48.556231 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.555747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5988b86fc-swgx5" event={"ID":"66a4ee05-3a27-42fd-880a-b6104d1a3b99","Type":"ContainerDied","Data":"9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158"}
Jul 07 17:57:48.556231 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.555796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5988b86fc-swgx5" event={"ID":"66a4ee05-3a27-42fd-880a-b6104d1a3b99","Type":"ContainerDied","Data":"493d52701961d3e29745dee2e460d9e52be2cef52596501062d9e52fc054b0dd"}
Jul 07 17:57:48.556231 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.555822    2580 scope.go:117] "RemoveContainer" containerID="9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158"
Jul 07 17:57:48.565246 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.565226    2580 scope.go:117] "RemoveContainer" containerID="9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158"
Jul 07 17:57:48.565503 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:57:48.565480    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158\": container with ID starting with 9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158 not found: ID does not exist" containerID="9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158"
Jul 07 17:57:48.565556 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.565519    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158"} err="failed to get container status \"9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158\": rpc error: code = NotFound desc = could not find container \"9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158\": container with ID starting with 9dbdb33b3ce7af380c3ac1f3eacae8ea7799ea290314ab47a962e409c4fd3158 not found: ID does not exist"
Jul 07 17:57:48.579918 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.579868    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5988b86fc-swgx5"]
Jul 07 17:57:48.583956 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.583930    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5988b86fc-swgx5"]
Jul 07 17:57:48.594498 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:57:48.594463    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a4ee05-3a27-42fd-880a-b6104d1a3b99" path="/var/lib/kubelet/pods/66a4ee05-3a27-42fd-880a-b6104d1a3b99/volumes"
Jul 07 17:58:37.087790 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.087751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-799f589db-t7p6j"]
Jul 07 17:58:37.088265 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.088103    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a4ee05-3a27-42fd-880a-b6104d1a3b99" containerName="console"
Jul 07 17:58:37.088265 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.088115    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a4ee05-3a27-42fd-880a-b6104d1a3b99" containerName="console"
Jul 07 17:58:37.088265 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.088175    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a4ee05-3a27-42fd-880a-b6104d1a3b99" containerName="console"
Jul 07 17:58:37.094282 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.094248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.107522 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.107488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-799f589db-t7p6j"]
Jul 07 17:58:37.150465 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lk6\" (UniqueName: \"kubernetes.io/projected/246813f9-a50d-4ded-bb9a-89c00bc7ea54-kube-api-access-67lk6\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.150660 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-service-ca\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.150660 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-oauth-serving-cert\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.150660 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-serving-cert\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.150660 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-oauth-config\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.150660 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-trusted-ca-bundle\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.150871 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.150678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-config\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.251838 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.251800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-config\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.251979 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.251863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lk6\" (UniqueName: \"kubernetes.io/projected/246813f9-a50d-4ded-bb9a-89c00bc7ea54-kube-api-access-67lk6\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.251979 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.251897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-service-ca\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.251979 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.251947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-oauth-serving-cert\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252180 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-serving-cert\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252259 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-oauth-config\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252310 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-trusted-ca-bundle\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252696 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252663    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-config\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252696 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252688    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-service-ca\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252839 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-oauth-serving-cert\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.252989 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.252972    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-trusted-ca-bundle\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.254828 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.254805    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-oauth-config\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.254926 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.254873    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-serving-cert\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.274754 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.274720    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lk6\" (UniqueName: \"kubernetes.io/projected/246813f9-a50d-4ded-bb9a-89c00bc7ea54-kube-api-access-67lk6\") pod \"console-799f589db-t7p6j\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") " pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.403353 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.403301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:37.542792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.542759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-799f589db-t7p6j"]
Jul 07 17:58:37.545663 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:58:37.545627    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246813f9_a50d_4ded_bb9a_89c00bc7ea54.slice/crio-a4350d67d4975f1b4c057fab15e24b9a178d813b036ecd88ab2b0e4b49ab2fd7 WatchSource:0}: Error finding container a4350d67d4975f1b4c057fab15e24b9a178d813b036ecd88ab2b0e4b49ab2fd7: Status 404 returned error can't find the container with id a4350d67d4975f1b4c057fab15e24b9a178d813b036ecd88ab2b0e4b49ab2fd7
Jul 07 17:58:37.730062 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.729963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799f589db-t7p6j" event={"ID":"246813f9-a50d-4ded-bb9a-89c00bc7ea54","Type":"ContainerStarted","Data":"3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2"}
Jul 07 17:58:37.730062 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:37.730015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799f589db-t7p6j" event={"ID":"246813f9-a50d-4ded-bb9a-89c00bc7ea54","Type":"ContainerStarted","Data":"a4350d67d4975f1b4c057fab15e24b9a178d813b036ecd88ab2b0e4b49ab2fd7"}
Jul 07 17:58:47.403627 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:47.403576    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:47.404221 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:47.403665    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:47.408555 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:47.408536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:47.447467 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:47.447413    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-799f589db-t7p6j" podStartSLOduration=10.447397283 podStartE2EDuration="10.447397283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:58:37.764431082 +0000 UTC m=+279.654791806" watchObservedRunningTime="2026-07-07 17:58:47.447397283 +0000 UTC m=+289.337758007"
Jul 07 17:58:47.769024 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:47.768951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 17:58:47.823262 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:47.823227    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f479448f-dvnxj"]
Jul 07 17:58:58.468922 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:58:58.468875    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 17:59:12.851031 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:12.850996    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68f479448f-dvnxj" podUID="0cfd7d9d-2e38-4c71-b503-d861c1891ee9" containerName="console" containerID="cri-o://ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f" gracePeriod=15
Jul 07 17:59:13.097090 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.097059    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:59:13.107812 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.107757    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"]
Jul 07 17:59:13.108040 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.108028    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cfd7d9d-2e38-4c71-b503-d861c1891ee9" containerName="console"
Jul 07 17:59:13.108083 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.108042    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cfd7d9d-2e38-4c71-b503-d861c1891ee9" containerName="console"
Jul 07 17:59:13.108116 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.108092    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cfd7d9d-2e38-4c71-b503-d861c1891ee9" containerName="console"
Jul 07 17:59:13.110835 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.110820    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.115667 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.115649    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 17:59:13.115792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.115727    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 17:59:13.115792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.115755    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4zd\""
Jul 07 17:59:13.121440 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.121417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"]
Jul 07 17:59:13.229284 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229249    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-service-ca\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229484 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229293    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-oauth-serving-cert\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229484 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-trusted-ca-bundle\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229484 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229345    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-serving-cert\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229484 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229438    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-525rs\" (UniqueName: \"kubernetes.io/projected/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-kube-api-access-525rs\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229484 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229472    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-config\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229745 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-oauth-config\") pod \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\" (UID: \"0cfd7d9d-2e38-4c71-b503-d861c1891ee9\") "
Jul 07 17:59:13.229745 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtb7\" (UniqueName: \"kubernetes.io/projected/640e96a6-79ac-42a0-baa1-8f759a029963-kube-api-access-jmtb7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.229849 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.229849 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229770    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:59:13.229849 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.229849 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229789    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-service-ca" (OuterVolumeSpecName: "service-ca") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:59:13.229849 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229832    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:59:13.230139 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229937    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-service-ca\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.230139 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229959    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-oauth-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.230139 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.229969    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-trusted-ca-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.230457 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.230431    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-config" (OuterVolumeSpecName: "console-config") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:59:13.231627 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.231600    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-kube-api-access-525rs" (OuterVolumeSpecName: "kube-api-access-525rs") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "kube-api-access-525rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:59:13.231727 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.231711    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:59:13.231773 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.231725    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0cfd7d9d-2e38-4c71-b503-d861c1891ee9" (UID: "0cfd7d9d-2e38-4c71-b503-d861c1891ee9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:59:13.331294 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.331294 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.331543 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtb7\" (UniqueName: \"kubernetes.io/projected/640e96a6-79ac-42a0-baa1-8f759a029963-kube-api-access-jmtb7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.331543 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331511    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-525rs\" (UniqueName: \"kubernetes.io/projected/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-kube-api-access-525rs\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.331543 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331524    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.331543 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331533    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-oauth-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.331543 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331543    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cfd7d9d-2e38-4c71-b503-d861c1891ee9-console-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:13.331773 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331617    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.331773 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.331696    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.341583 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.341559    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtb7\" (UniqueName: \"kubernetes.io/projected/640e96a6-79ac-42a0-baa1-8f759a029963-kube-api-access-jmtb7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.418861 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.418778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:13.556565 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.556418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"]
Jul 07 17:59:13.559225 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:13.559197    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640e96a6_79ac_42a0_baa1_8f759a029963.slice/crio-70b8575960a852c1cc0d6d581613a007e04e01ad7c4e6a2e06bb1f16899ba7dc WatchSource:0}: Error finding container 70b8575960a852c1cc0d6d581613a007e04e01ad7c4e6a2e06bb1f16899ba7dc: Status 404 returned error can't find the container with id 70b8575960a852c1cc0d6d581613a007e04e01ad7c4e6a2e06bb1f16899ba7dc
Jul 07 17:59:13.561061 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.561035    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 17:59:13.855975 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.855941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk" event={"ID":"640e96a6-79ac-42a0-baa1-8f759a029963","Type":"ContainerStarted","Data":"70b8575960a852c1cc0d6d581613a007e04e01ad7c4e6a2e06bb1f16899ba7dc"}
Jul 07 17:59:13.857094 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.857059    2580 generic.go:356] "Generic (PLEG): container finished" podID="0cfd7d9d-2e38-4c71-b503-d861c1891ee9" containerID="ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f" exitCode=0
Jul 07 17:59:13.857196 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.857105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f479448f-dvnxj" event={"ID":"0cfd7d9d-2e38-4c71-b503-d861c1891ee9","Type":"ContainerDied","Data":"ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f"}
Jul 07 17:59:13.857196 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.857117    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f479448f-dvnxj"
Jul 07 17:59:13.857196 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.857178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f479448f-dvnxj" event={"ID":"0cfd7d9d-2e38-4c71-b503-d861c1891ee9","Type":"ContainerDied","Data":"cb81b1f0d037c4e740cf6e9d14071bdaa92364475b6aebfc5607f6617eeaf350"}
Jul 07 17:59:13.857292 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.857207    2580 scope.go:117] "RemoveContainer" containerID="ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f"
Jul 07 17:59:13.870788 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.870770    2580 scope.go:117] "RemoveContainer" containerID="ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f"
Jul 07 17:59:13.871075 ip-10-0-134-243 kubenswrapper[2580]: E0707 17:59:13.871059    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f\": container with ID starting with ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f not found: ID does not exist" containerID="ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f"
Jul 07 17:59:13.871128 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.871083    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f"} err="failed to get container status \"ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f\": rpc error: code = NotFound desc = could not find container \"ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f\": container with ID starting with ebc89357ade8dbb127ce7ff5d8518c3c1a0fcbfc0305b2a04eb26e35760a084f not found: ID does not exist"
Jul 07 17:59:13.883222 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.883198    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f479448f-dvnxj"]
Jul 07 17:59:13.887339 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:13.887317    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68f479448f-dvnxj"]
Jul 07 17:59:14.595245 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:14.595213    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cfd7d9d-2e38-4c71-b503-d861c1891ee9" path="/var/lib/kubelet/pods/0cfd7d9d-2e38-4c71-b503-d861c1891ee9/volumes"
Jul 07 17:59:21.894299 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:21.894264    2580 generic.go:356] "Generic (PLEG): container finished" podID="640e96a6-79ac-42a0-baa1-8f759a029963" containerID="5037992b2730664f474fa5b801d2ef126301400ed4fc294ad751fce62f93f524" exitCode=0
Jul 07 17:59:21.894703 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:21.894342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk" event={"ID":"640e96a6-79ac-42a0-baa1-8f759a029963","Type":"ContainerDied","Data":"5037992b2730664f474fa5b801d2ef126301400ed4fc294ad751fce62f93f524"}
Jul 07 17:59:23.904003 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:23.903973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk" event={"ID":"640e96a6-79ac-42a0-baa1-8f759a029963","Type":"ContainerStarted","Data":"85f7fcab502756f167cbee4d4b28488a15ee6e3569df5d6c701336b0eac00403"}
Jul 07 17:59:24.908975 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:24.908941    2580 generic.go:356] "Generic (PLEG): container finished" podID="640e96a6-79ac-42a0-baa1-8f759a029963" containerID="85f7fcab502756f167cbee4d4b28488a15ee6e3569df5d6c701336b0eac00403" exitCode=0
Jul 07 17:59:24.909356 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:24.909024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk" event={"ID":"640e96a6-79ac-42a0-baa1-8f759a029963","Type":"ContainerDied","Data":"85f7fcab502756f167cbee4d4b28488a15ee6e3569df5d6c701336b0eac00403"}
Jul 07 17:59:33.947125 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:33.947089    2580 generic.go:356] "Generic (PLEG): container finished" podID="640e96a6-79ac-42a0-baa1-8f759a029963" containerID="b9774b9fe794c22f9d723174233cb8a868d2b6b22a4aa44e92c0920323e10dac" exitCode=0
Jul 07 17:59:33.947125 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:33.947132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk" event={"ID":"640e96a6-79ac-42a0-baa1-8f759a029963","Type":"ContainerDied","Data":"b9774b9fe794c22f9d723174233cb8a868d2b6b22a4aa44e92c0920323e10dac"}
Jul 07 17:59:35.118144 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.118105    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:35.205635 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.205539    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-util\") pod \"640e96a6-79ac-42a0-baa1-8f759a029963\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") "
Jul 07 17:59:35.205635 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.205606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-bundle\") pod \"640e96a6-79ac-42a0-baa1-8f759a029963\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") "
Jul 07 17:59:35.205635 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.205631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmtb7\" (UniqueName: \"kubernetes.io/projected/640e96a6-79ac-42a0-baa1-8f759a029963-kube-api-access-jmtb7\") pod \"640e96a6-79ac-42a0-baa1-8f759a029963\" (UID: \"640e96a6-79ac-42a0-baa1-8f759a029963\") "
Jul 07 17:59:35.206455 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.206424    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-bundle" (OuterVolumeSpecName: "bundle") pod "640e96a6-79ac-42a0-baa1-8f759a029963" (UID: "640e96a6-79ac-42a0-baa1-8f759a029963"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:59:35.208011 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.207986    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/640e96a6-79ac-42a0-baa1-8f759a029963-kube-api-access-jmtb7" (OuterVolumeSpecName: "kube-api-access-jmtb7") pod "640e96a6-79ac-42a0-baa1-8f759a029963" (UID: "640e96a6-79ac-42a0-baa1-8f759a029963"). InnerVolumeSpecName "kube-api-access-jmtb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:59:35.210376 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.210354    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-util" (OuterVolumeSpecName: "util") pod "640e96a6-79ac-42a0-baa1-8f759a029963" (UID: "640e96a6-79ac-42a0-baa1-8f759a029963"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:59:35.306516 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.306482    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:35.306516 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.306509    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmtb7\" (UniqueName: \"kubernetes.io/projected/640e96a6-79ac-42a0-baa1-8f759a029963-kube-api-access-jmtb7\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:35.306516 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.306520    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/640e96a6-79ac-42a0-baa1-8f759a029963-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:35.956015 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.955977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk" event={"ID":"640e96a6-79ac-42a0-baa1-8f759a029963","Type":"ContainerDied","Data":"70b8575960a852c1cc0d6d581613a007e04e01ad7c4e6a2e06bb1f16899ba7dc"}
Jul 07 17:59:35.956184 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.956030    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70b8575960a852c1cc0d6d581613a007e04e01ad7c4e6a2e06bb1f16899ba7dc"
Jul 07 17:59:35.956184 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:35.956036    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nt4jk"
Jul 07 17:59:41.814251 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814217    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"]
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814484    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="extract"
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814495    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="extract"
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814522    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="pull"
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814529    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="pull"
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814536    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="util"
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814542    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="util"
Jul 07 17:59:41.814623 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.814582    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="640e96a6-79ac-42a0-baa1-8f759a029963" containerName="extract"
Jul 07 17:59:41.816469 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.816455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:41.822897 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.822874    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 07 17:59:41.823041 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.822955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xntm8\""
Jul 07 17:59:41.823193 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.823178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:59:41.835657 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.835630    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"]
Jul 07 17:59:41.855528 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.855479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfn96\" (UniqueName: \"kubernetes.io/projected/c741cfb2-1079-4fd1-97fd-6720062451ba-kube-api-access-sfn96\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq4qm\" (UID: \"c741cfb2-1079-4fd1-97fd-6720062451ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:41.855685 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.855543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c741cfb2-1079-4fd1-97fd-6720062451ba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq4qm\" (UID: \"c741cfb2-1079-4fd1-97fd-6720062451ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:41.956550 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.956502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfn96\" (UniqueName: \"kubernetes.io/projected/c741cfb2-1079-4fd1-97fd-6720062451ba-kube-api-access-sfn96\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq4qm\" (UID: \"c741cfb2-1079-4fd1-97fd-6720062451ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:41.956550 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.956555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c741cfb2-1079-4fd1-97fd-6720062451ba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq4qm\" (UID: \"c741cfb2-1079-4fd1-97fd-6720062451ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:41.956970 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.956955    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c741cfb2-1079-4fd1-97fd-6720062451ba-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq4qm\" (UID: \"c741cfb2-1079-4fd1-97fd-6720062451ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:41.967083 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:41.967056    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfn96\" (UniqueName: \"kubernetes.io/projected/c741cfb2-1079-4fd1-97fd-6720062451ba-kube-api-access-sfn96\") pod \"cert-manager-operator-controller-manager-55c444bf86-rq4qm\" (UID: \"c741cfb2-1079-4fd1-97fd-6720062451ba\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:42.125475 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:42.125446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"
Jul 07 17:59:42.263804 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:42.263772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm"]
Jul 07 17:59:42.267121 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:42.267091    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc741cfb2_1079_4fd1_97fd_6720062451ba.slice/crio-35c0ccc974b0d2e2c1a2dad2556cfe8437d1aec6b496fa0bc8f851e01c2f9e80 WatchSource:0}: Error finding container 35c0ccc974b0d2e2c1a2dad2556cfe8437d1aec6b496fa0bc8f851e01c2f9e80: Status 404 returned error can't find the container with id 35c0ccc974b0d2e2c1a2dad2556cfe8437d1aec6b496fa0bc8f851e01c2f9e80
Jul 07 17:59:42.983183 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:42.983143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm" event={"ID":"c741cfb2-1079-4fd1-97fd-6720062451ba","Type":"ContainerStarted","Data":"35c0ccc974b0d2e2c1a2dad2556cfe8437d1aec6b496fa0bc8f851e01c2f9e80"}
Jul 07 17:59:44.993721 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:44.993682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm" event={"ID":"c741cfb2-1079-4fd1-97fd-6720062451ba","Type":"ContainerStarted","Data":"ec8f85cf872c10761ba46178830c52849bb73cf39749be25fc7fb781a9373f2c"}
Jul 07 17:59:45.071259 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.071201    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rq4qm" podStartSLOduration=2.222563148 podStartE2EDuration="4.071180526s" totalImagesPullingTime="1.848617378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:59:45.045936497 +0000 UTC m=+346.936297222" watchObservedRunningTime="2026-07-07 17:59:45.071180526 +0000 UTC m=+346.961541249"
Jul 07 17:59:45.295322 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.295239    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wpvjs"]
Jul 07 17:59:45.297389 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.297371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.305561 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.305538    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 17:59:45.305701 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.305683    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 17:59:45.307346 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.307333    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jzjfg\""
Jul 07 17:59:45.326460 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.326405    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wpvjs"]
Jul 07 17:59:45.381444 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.381400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdmmn\" (UniqueName: \"kubernetes.io/projected/77409fcf-9a3d-4c05-adc9-b93165f1e1e5-kube-api-access-vdmmn\") pod \"cert-manager-webhook-6486ffd997-wpvjs\" (UID: \"77409fcf-9a3d-4c05-adc9-b93165f1e1e5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.381624 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.381458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77409fcf-9a3d-4c05-adc9-b93165f1e1e5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wpvjs\" (UID: \"77409fcf-9a3d-4c05-adc9-b93165f1e1e5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.481830 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.481791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77409fcf-9a3d-4c05-adc9-b93165f1e1e5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wpvjs\" (UID: \"77409fcf-9a3d-4c05-adc9-b93165f1e1e5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.482025 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.481894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdmmn\" (UniqueName: \"kubernetes.io/projected/77409fcf-9a3d-4c05-adc9-b93165f1e1e5-kube-api-access-vdmmn\") pod \"cert-manager-webhook-6486ffd997-wpvjs\" (UID: \"77409fcf-9a3d-4c05-adc9-b93165f1e1e5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.512549 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.512510    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/77409fcf-9a3d-4c05-adc9-b93165f1e1e5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wpvjs\" (UID: \"77409fcf-9a3d-4c05-adc9-b93165f1e1e5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.517082 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.517058    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdmmn\" (UniqueName: \"kubernetes.io/projected/77409fcf-9a3d-4c05-adc9-b93165f1e1e5-kube-api-access-vdmmn\") pod \"cert-manager-webhook-6486ffd997-wpvjs\" (UID: \"77409fcf-9a3d-4c05-adc9-b93165f1e1e5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.606213 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.606180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:45.793533 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.793503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wpvjs"]
Jul 07 17:59:45.798871 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:45.798842    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77409fcf_9a3d_4c05_adc9_b93165f1e1e5.slice/crio-2d83e979ccf31f449cf4cc6e05f4156f5a3737a9328ac2a15bcbac31c5987ab3 WatchSource:0}: Error finding container 2d83e979ccf31f449cf4cc6e05f4156f5a3737a9328ac2a15bcbac31c5987ab3: Status 404 returned error can't find the container with id 2d83e979ccf31f449cf4cc6e05f4156f5a3737a9328ac2a15bcbac31c5987ab3
Jul 07 17:59:45.931792 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.931712    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"]
Jul 07 17:59:45.934822 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.934804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:45.953166 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.953137    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4zd\""
Jul 07 17:59:45.953317 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.953134    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 17:59:45.953317 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.953137    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 17:59:45.986222 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.986194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:45.986396 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.986244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcdm5\" (UniqueName: \"kubernetes.io/projected/443bb450-9486-41f4-9268-66beab59a620-kube-api-access-tcdm5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:45.986396 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.986335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:45.999101 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:45.999067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs" event={"ID":"77409fcf-9a3d-4c05-adc9-b93165f1e1e5","Type":"ContainerStarted","Data":"2d83e979ccf31f449cf4cc6e05f4156f5a3737a9328ac2a15bcbac31c5987ab3"}
Jul 07 17:59:46.048428 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.048394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"]
Jul 07 17:59:46.086848 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.086808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcdm5\" (UniqueName: \"kubernetes.io/projected/443bb450-9486-41f4-9268-66beab59a620-kube-api-access-tcdm5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.087068 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.087011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.087068 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.087061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.087384 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.087364    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.087427 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.087396    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.127447 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.127412    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcdm5\" (UniqueName: \"kubernetes.io/projected/443bb450-9486-41f4-9268-66beab59a620-kube-api-access-tcdm5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.243955 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.243846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:46.389532 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:46.389496    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"]
Jul 07 17:59:46.393737 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:46.393710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443bb450_9486_41f4_9268_66beab59a620.slice/crio-8d34523aafdaeae1c155aa76f6e721aaf4a14ec6210c73c2fd29b9c9957e8c1b WatchSource:0}: Error finding container 8d34523aafdaeae1c155aa76f6e721aaf4a14ec6210c73c2fd29b9c9957e8c1b: Status 404 returned error can't find the container with id 8d34523aafdaeae1c155aa76f6e721aaf4a14ec6210c73c2fd29b9c9957e8c1b
Jul 07 17:59:47.005559 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:47.005527    2580 generic.go:356] "Generic (PLEG): container finished" podID="443bb450-9486-41f4-9268-66beab59a620" containerID="6188d532c189dcf8331ac1299dd219ec2b5492fb57c7d9d9e665f01d704153c8" exitCode=0
Jul 07 17:59:47.006072 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:47.005613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq" event={"ID":"443bb450-9486-41f4-9268-66beab59a620","Type":"ContainerDied","Data":"6188d532c189dcf8331ac1299dd219ec2b5492fb57c7d9d9e665f01d704153c8"}
Jul 07 17:59:47.006072 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:47.005680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq" event={"ID":"443bb450-9486-41f4-9268-66beab59a620","Type":"ContainerStarted","Data":"8d34523aafdaeae1c155aa76f6e721aaf4a14ec6210c73c2fd29b9c9957e8c1b"}
Jul 07 17:59:50.021897 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:50.021863    2580 generic.go:356] "Generic (PLEG): container finished" podID="443bb450-9486-41f4-9268-66beab59a620" containerID="a06ea51afa1b9160ea60a83130b96d0e1cf6bf8695707c118c7aebd4fcbecc89" exitCode=0
Jul 07 17:59:50.022311 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:50.021955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq" event={"ID":"443bb450-9486-41f4-9268-66beab59a620","Type":"ContainerDied","Data":"a06ea51afa1b9160ea60a83130b96d0e1cf6bf8695707c118c7aebd4fcbecc89"}
Jul 07 17:59:50.023612 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:50.023587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs" event={"ID":"77409fcf-9a3d-4c05-adc9-b93165f1e1e5","Type":"ContainerStarted","Data":"5eea5ac6961d8aba3de816dd45a1c9c479cf64a5fafe11a9af3048465b959adc"}
Jul 07 17:59:50.023737 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:50.023710    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:50.062354 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:50.062306    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs" podStartSLOduration=1.505249693 podStartE2EDuration="5.06228961s" totalImagesPullingTime="3.557039917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:59:50.06143551 +0000 UTC m=+351.951796238" watchObservedRunningTime="2026-07-07 17:59:50.06228961 +0000 UTC m=+351.952650334"
Jul 07 17:59:51.030021 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.029986    2580 generic.go:356] "Generic (PLEG): container finished" podID="443bb450-9486-41f4-9268-66beab59a620" containerID="85c2b8c6746fd6a726819b7d66a34009978b300c7d0660f389e033174311e919" exitCode=0
Jul 07 17:59:51.030523 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.030079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq" event={"ID":"443bb450-9486-41f4-9268-66beab59a620","Type":"ContainerDied","Data":"85c2b8c6746fd6a726819b7d66a34009978b300c7d0660f389e033174311e919"}
Jul 07 17:59:51.700008 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.699977    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"]
Jul 07 17:59:51.703828 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.703808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:51.706937 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.706896    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ssbmh\""
Jul 07 17:59:51.715195 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.715173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"]
Jul 07 17:59:51.733549 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.733502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7edc6e18-870a-4cfa-974c-338f877d8f9a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-njbmd\" (UID: \"7edc6e18-870a-4cfa-974c-338f877d8f9a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:51.733549 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.733558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjmc6\" (UniqueName: \"kubernetes.io/projected/7edc6e18-870a-4cfa-974c-338f877d8f9a-kube-api-access-bjmc6\") pod \"cert-manager-cainjector-559cdd8c87-njbmd\" (UID: \"7edc6e18-870a-4cfa-974c-338f877d8f9a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:51.834467 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.834422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7edc6e18-870a-4cfa-974c-338f877d8f9a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-njbmd\" (UID: \"7edc6e18-870a-4cfa-974c-338f877d8f9a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:51.834467 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.834470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjmc6\" (UniqueName: \"kubernetes.io/projected/7edc6e18-870a-4cfa-974c-338f877d8f9a-kube-api-access-bjmc6\") pod \"cert-manager-cainjector-559cdd8c87-njbmd\" (UID: \"7edc6e18-870a-4cfa-974c-338f877d8f9a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:51.843493 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.843462    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7edc6e18-870a-4cfa-974c-338f877d8f9a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-njbmd\" (UID: \"7edc6e18-870a-4cfa-974c-338f877d8f9a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:51.843658 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:51.843640    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjmc6\" (UniqueName: \"kubernetes.io/projected/7edc6e18-870a-4cfa-974c-338f877d8f9a-kube-api-access-bjmc6\") pod \"cert-manager-cainjector-559cdd8c87-njbmd\" (UID: \"7edc6e18-870a-4cfa-974c-338f877d8f9a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:52.012526 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.012441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"
Jul 07 17:59:52.163612 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.163581    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:52.164076 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.164057    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-njbmd"]
Jul 07 17:59:52.166864 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:52.166841    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7edc6e18_870a_4cfa_974c_338f877d8f9a.slice/crio-3095866b4a97630360ec632476d62fd65d57a1f4ff5826d8af5ca7d9368fdc8c WatchSource:0}: Error finding container 3095866b4a97630360ec632476d62fd65d57a1f4ff5826d8af5ca7d9368fdc8c: Status 404 returned error can't find the container with id 3095866b4a97630360ec632476d62fd65d57a1f4ff5826d8af5ca7d9368fdc8c
Jul 07 17:59:52.237595 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.237566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-bundle\") pod \"443bb450-9486-41f4-9268-66beab59a620\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") "
Jul 07 17:59:52.237753 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.237629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcdm5\" (UniqueName: \"kubernetes.io/projected/443bb450-9486-41f4-9268-66beab59a620-kube-api-access-tcdm5\") pod \"443bb450-9486-41f4-9268-66beab59a620\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") "
Jul 07 17:59:52.237753 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.237662    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-util\") pod \"443bb450-9486-41f4-9268-66beab59a620\" (UID: \"443bb450-9486-41f4-9268-66beab59a620\") "
Jul 07 17:59:52.238042 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.238017    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-bundle" (OuterVolumeSpecName: "bundle") pod "443bb450-9486-41f4-9268-66beab59a620" (UID: "443bb450-9486-41f4-9268-66beab59a620"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:59:52.240147 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.240122    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/443bb450-9486-41f4-9268-66beab59a620-kube-api-access-tcdm5" (OuterVolumeSpecName: "kube-api-access-tcdm5") pod "443bb450-9486-41f4-9268-66beab59a620" (UID: "443bb450-9486-41f4-9268-66beab59a620"). InnerVolumeSpecName "kube-api-access-tcdm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:59:52.242785 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.242760    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-util" (OuterVolumeSpecName: "util") pod "443bb450-9486-41f4-9268-66beab59a620" (UID: "443bb450-9486-41f4-9268-66beab59a620"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:59:52.338744 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.338658    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcdm5\" (UniqueName: \"kubernetes.io/projected/443bb450-9486-41f4-9268-66beab59a620-kube-api-access-tcdm5\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:52.338744 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.338689    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:52.338744 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:52.338700    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/443bb450-9486-41f4-9268-66beab59a620-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 17:59:53.040867 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:53.040791    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq"
Jul 07 17:59:53.040867 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:53.040813    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpj5tq" event={"ID":"443bb450-9486-41f4-9268-66beab59a620","Type":"ContainerDied","Data":"8d34523aafdaeae1c155aa76f6e721aaf4a14ec6210c73c2fd29b9c9957e8c1b"}
Jul 07 17:59:53.041095 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:53.040891    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d34523aafdaeae1c155aa76f6e721aaf4a14ec6210c73c2fd29b9c9957e8c1b"
Jul 07 17:59:53.042535 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:53.042511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd" event={"ID":"7edc6e18-870a-4cfa-974c-338f877d8f9a","Type":"ContainerStarted","Data":"4b3827d46668771d6108005aa20311644b590275b45b1ad74bcc15fa7c858542"}
Jul 07 17:59:53.042661 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:53.042542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd" event={"ID":"7edc6e18-870a-4cfa-974c-338f877d8f9a","Type":"ContainerStarted","Data":"3095866b4a97630360ec632476d62fd65d57a1f4ff5826d8af5ca7d9368fdc8c"}
Jul 07 17:59:53.061051 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:53.060978    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-njbmd" podStartSLOduration=2.060959006 podStartE2EDuration="2.060959006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:59:53.060352878 +0000 UTC m=+354.950713603" watchObservedRunningTime="2026-07-07 17:59:53.060959006 +0000 UTC m=+354.951319732"
Jul 07 17:59:55.698273 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698238    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pdx4g"]
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698658    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="util"
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698677    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="util"
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="extract"
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698708    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="extract"
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698728    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="pull"
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698736    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="pull"
Jul 07 17:59:55.698843 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.698811    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="443bb450-9486-41f4-9268-66beab59a620" containerName="extract"
Jul 07 17:59:55.702307 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.702289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:55.705285 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.705263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ml8h4\""
Jul 07 17:59:55.713243 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.713222    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pdx4g"]
Jul 07 17:59:55.769136 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.769095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md9sd\" (UniqueName: \"kubernetes.io/projected/65122b46-6e23-473a-bb18-b68b73af217a-kube-api-access-md9sd\") pod \"cert-manager-55b4d954d5-pdx4g\" (UID: \"65122b46-6e23-473a-bb18-b68b73af217a\") " pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:55.769317 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.769156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65122b46-6e23-473a-bb18-b68b73af217a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pdx4g\" (UID: \"65122b46-6e23-473a-bb18-b68b73af217a\") " pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:55.870508 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.870472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md9sd\" (UniqueName: \"kubernetes.io/projected/65122b46-6e23-473a-bb18-b68b73af217a-kube-api-access-md9sd\") pod \"cert-manager-55b4d954d5-pdx4g\" (UID: \"65122b46-6e23-473a-bb18-b68b73af217a\") " pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:55.870678 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.870516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65122b46-6e23-473a-bb18-b68b73af217a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pdx4g\" (UID: \"65122b46-6e23-473a-bb18-b68b73af217a\") " pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:55.880749 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.880724    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md9sd\" (UniqueName: \"kubernetes.io/projected/65122b46-6e23-473a-bb18-b68b73af217a-kube-api-access-md9sd\") pod \"cert-manager-55b4d954d5-pdx4g\" (UID: \"65122b46-6e23-473a-bb18-b68b73af217a\") " pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:55.881826 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:55.881801    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65122b46-6e23-473a-bb18-b68b73af217a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pdx4g\" (UID: \"65122b46-6e23-473a-bb18-b68b73af217a\") " pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:56.011349 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:56.011238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pdx4g"
Jul 07 17:59:56.032117 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:56.032090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wpvjs"
Jul 07 17:59:56.146291 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:56.146262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pdx4g"]
Jul 07 17:59:56.148342 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:56.148310    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65122b46_6e23_473a_bb18_b68b73af217a.slice/crio-5c422a3d82489268ae430dc9b0a25e8c5f403b9c01c054f66cae21ffdd0a4252 WatchSource:0}: Error finding container 5c422a3d82489268ae430dc9b0a25e8c5f403b9c01c054f66cae21ffdd0a4252: Status 404 returned error can't find the container with id 5c422a3d82489268ae430dc9b0a25e8c5f403b9c01c054f66cae21ffdd0a4252
Jul 07 17:59:57.061788 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:57.061747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pdx4g" event={"ID":"65122b46-6e23-473a-bb18-b68b73af217a","Type":"ContainerStarted","Data":"e7df8ed636cdc81b40bc7b8d2a00564327b15d695e31bc3e237864ea81c07e28"}
Jul 07 17:59:57.061788 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:57.061794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pdx4g" event={"ID":"65122b46-6e23-473a-bb18-b68b73af217a","Type":"ContainerStarted","Data":"5c422a3d82489268ae430dc9b0a25e8c5f403b9c01c054f66cae21ffdd0a4252"}
Jul 07 17:59:57.082130 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:57.082076    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-pdx4g" podStartSLOduration=2.082059845 podStartE2EDuration="2.082059845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:59:57.08067075 +0000 UTC m=+358.971031466" watchObservedRunningTime="2026-07-07 17:59:57.082059845 +0000 UTC m=+358.972420569"
Jul 07 17:59:59.355772 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.355732    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"]
Jul 07 17:59:59.395685 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.395647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"]
Jul 07 17:59:59.395836 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.395751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.398953 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.398932    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:59:59.399038 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.399010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 17:59:59.399847 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.399833    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lxh5j\""
Jul 07 17:59:59.499523 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.499484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168475b4-ba86-490f-99d7-5816faa38363-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5p9kx\" (UID: \"168475b4-ba86-490f-99d7-5816faa38363\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.499702 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.499533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgxnr\" (UniqueName: \"kubernetes.io/projected/168475b4-ba86-490f-99d7-5816faa38363-kube-api-access-pgxnr\") pod \"openshift-lws-operator-fd8ccff4c-5p9kx\" (UID: \"168475b4-ba86-490f-99d7-5816faa38363\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.600456 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.600426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168475b4-ba86-490f-99d7-5816faa38363-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5p9kx\" (UID: \"168475b4-ba86-490f-99d7-5816faa38363\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.600609 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.600468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgxnr\" (UniqueName: \"kubernetes.io/projected/168475b4-ba86-490f-99d7-5816faa38363-kube-api-access-pgxnr\") pod \"openshift-lws-operator-fd8ccff4c-5p9kx\" (UID: \"168475b4-ba86-490f-99d7-5816faa38363\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.600819 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.600798    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/168475b4-ba86-490f-99d7-5816faa38363-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5p9kx\" (UID: \"168475b4-ba86-490f-99d7-5816faa38363\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.609674 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.609622    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgxnr\" (UniqueName: \"kubernetes.io/projected/168475b4-ba86-490f-99d7-5816faa38363-kube-api-access-pgxnr\") pod \"openshift-lws-operator-fd8ccff4c-5p9kx\" (UID: \"168475b4-ba86-490f-99d7-5816faa38363\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.704752 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.704715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"
Jul 07 17:59:59.861593 ip-10-0-134-243 kubenswrapper[2580]: I0707 17:59:59.861504    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx"]
Jul 07 17:59:59.864801 ip-10-0-134-243 kubenswrapper[2580]: W0707 17:59:59.864772    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod168475b4_ba86_490f_99d7_5816faa38363.slice/crio-248a638a8f4781c8320458d1a3329b063ab833c3292da3783dbeabaf1e9d9bbe WatchSource:0}: Error finding container 248a638a8f4781c8320458d1a3329b063ab833c3292da3783dbeabaf1e9d9bbe: Status 404 returned error can't find the container with id 248a638a8f4781c8320458d1a3329b063ab833c3292da3783dbeabaf1e9d9bbe
Jul 07 18:00:00.076131 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:00.076096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx" event={"ID":"168475b4-ba86-490f-99d7-5816faa38363","Type":"ContainerStarted","Data":"248a638a8f4781c8320458d1a3329b063ab833c3292da3783dbeabaf1e9d9bbe"}
Jul 07 18:00:04.095742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:04.095694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx" event={"ID":"168475b4-ba86-490f-99d7-5816faa38363","Type":"ContainerStarted","Data":"3dc8cf63fa1b3b209b8e27990d15225f7acdfba4ebc97fa7252f7312d80d9975"}
Jul 07 18:00:04.116156 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:04.115955    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5p9kx" podStartSLOduration=1.326632126 podStartE2EDuration="5.115936008s" totalImagesPullingTime="3.789303882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:04.115013089 +0000 UTC m=+366.005373825" watchObservedRunningTime="2026-07-07 18:00:04.115936008 +0000 UTC m=+366.006296724"
Jul 07 18:00:06.703131 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.703095    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"]
Jul 07 18:00:06.705599 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.705582    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.709138 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.709119    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4zd\""
Jul 07 18:00:06.709265 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.709141    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 18:00:06.709805 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.709788    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 18:00:06.717561 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.717533    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"]
Jul 07 18:00:06.764376 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.764336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxtwr\" (UniqueName: \"kubernetes.io/projected/6051b35d-15fc-43ac-b8e2-31a2996772f7-kube-api-access-kxtwr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.764589 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.764398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.764589 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.764496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.865205 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.865161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.865393 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.865231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.865393 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.865293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtwr\" (UniqueName: \"kubernetes.io/projected/6051b35d-15fc-43ac-b8e2-31a2996772f7-kube-api-access-kxtwr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.865618 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.865595    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.865688 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.865609    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:06.877220 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:06.877191    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxtwr\" (UniqueName: \"kubernetes.io/projected/6051b35d-15fc-43ac-b8e2-31a2996772f7-kube-api-access-kxtwr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:07.014926 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:07.014813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:07.149885 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:07.149860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"]
Jul 07 18:00:07.152520 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:07.152484    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6051b35d_15fc_43ac_b8e2_31a2996772f7.slice/crio-ff27733bc91e84181c708ef232a4df7c600d1a131a4142ce354718e6f4009bd3 WatchSource:0}: Error finding container ff27733bc91e84181c708ef232a4df7c600d1a131a4142ce354718e6f4009bd3: Status 404 returned error can't find the container with id ff27733bc91e84181c708ef232a4df7c600d1a131a4142ce354718e6f4009bd3
Jul 07 18:00:08.117127 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:08.117084    2580 generic.go:356] "Generic (PLEG): container finished" podID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerID="bb487b43397257c5c5ec233f4d283d07a6ae7e055850b821aa506c4fca418b50" exitCode=0
Jul 07 18:00:08.117567 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:08.117171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx" event={"ID":"6051b35d-15fc-43ac-b8e2-31a2996772f7","Type":"ContainerDied","Data":"bb487b43397257c5c5ec233f4d283d07a6ae7e055850b821aa506c4fca418b50"}
Jul 07 18:00:08.117567 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:08.117225    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx" event={"ID":"6051b35d-15fc-43ac-b8e2-31a2996772f7","Type":"ContainerStarted","Data":"ff27733bc91e84181c708ef232a4df7c600d1a131a4142ce354718e6f4009bd3"}
Jul 07 18:00:09.123849 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:09.123817    2580 generic.go:356] "Generic (PLEG): container finished" podID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerID="325f674646c16df2a8e51856f626ffc05b17482b1e1c739f287c583ce4493faa" exitCode=0
Jul 07 18:00:09.124243 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:09.123934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx" event={"ID":"6051b35d-15fc-43ac-b8e2-31a2996772f7","Type":"ContainerDied","Data":"325f674646c16df2a8e51856f626ffc05b17482b1e1c739f287c583ce4493faa"}
Jul 07 18:00:10.131491 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:10.131456    2580 generic.go:356] "Generic (PLEG): container finished" podID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerID="8186fef859216eeb7d0fb878c92a51122707df0a04a09d1a8504c9f1022bc382" exitCode=0
Jul 07 18:00:10.131868 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:10.131499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx" event={"ID":"6051b35d-15fc-43ac-b8e2-31a2996772f7","Type":"ContainerDied","Data":"8186fef859216eeb7d0fb878c92a51122707df0a04a09d1a8504c9f1022bc382"}
Jul 07 18:00:11.261326 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.261294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:11.404990 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.404857    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-util\") pod \"6051b35d-15fc-43ac-b8e2-31a2996772f7\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") "
Jul 07 18:00:11.404990 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.404966    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-bundle\") pod \"6051b35d-15fc-43ac-b8e2-31a2996772f7\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") "
Jul 07 18:00:11.405186 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.405035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxtwr\" (UniqueName: \"kubernetes.io/projected/6051b35d-15fc-43ac-b8e2-31a2996772f7-kube-api-access-kxtwr\") pod \"6051b35d-15fc-43ac-b8e2-31a2996772f7\" (UID: \"6051b35d-15fc-43ac-b8e2-31a2996772f7\") "
Jul 07 18:00:11.405675 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.405641    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-bundle" (OuterVolumeSpecName: "bundle") pod "6051b35d-15fc-43ac-b8e2-31a2996772f7" (UID: "6051b35d-15fc-43ac-b8e2-31a2996772f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:00:11.407257 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.407224    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6051b35d-15fc-43ac-b8e2-31a2996772f7-kube-api-access-kxtwr" (OuterVolumeSpecName: "kube-api-access-kxtwr") pod "6051b35d-15fc-43ac-b8e2-31a2996772f7" (UID: "6051b35d-15fc-43ac-b8e2-31a2996772f7"). InnerVolumeSpecName "kube-api-access-kxtwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:00:11.410586 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.410565    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-util" (OuterVolumeSpecName: "util") pod "6051b35d-15fc-43ac-b8e2-31a2996772f7" (UID: "6051b35d-15fc-43ac-b8e2-31a2996772f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:00:11.505946 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.505868    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:00:11.505946 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.505937    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6051b35d-15fc-43ac-b8e2-31a2996772f7-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:00:11.505946 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:11.505947    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxtwr\" (UniqueName: \"kubernetes.io/projected/6051b35d-15fc-43ac-b8e2-31a2996772f7-kube-api-access-kxtwr\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:00:12.143355 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:12.143314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx" event={"ID":"6051b35d-15fc-43ac-b8e2-31a2996772f7","Type":"ContainerDied","Data":"ff27733bc91e84181c708ef232a4df7c600d1a131a4142ce354718e6f4009bd3"}
Jul 07 18:00:12.143355 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:12.143364    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff27733bc91e84181c708ef232a4df7c600d1a131a4142ce354718e6f4009bd3"
Jul 07 18:00:12.143563 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:12.143361    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgz7tx"
Jul 07 18:00:20.677289 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677257    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"]
Jul 07 18:00:20.677827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677708    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="pull"
Jul 07 18:00:20.677827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677727    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="pull"
Jul 07 18:00:20.677827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677749    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="extract"
Jul 07 18:00:20.677827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677758    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="extract"
Jul 07 18:00:20.677827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677770    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="util"
Jul 07 18:00:20.677827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677778    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="util"
Jul 07 18:00:20.678116 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.677850    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6051b35d-15fc-43ac-b8e2-31a2996772f7" containerName="extract"
Jul 07 18:00:20.680022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.680004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.683353 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.683327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 18:00:20.683472 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.683379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 18:00:20.683472 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.683454    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 18:00:20.683713 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.683691    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 18:00:20.683835 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.683727    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sp2sj\""
Jul 07 18:00:20.705657 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.705619    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"]
Jul 07 18:00:20.781477 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.781445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0727af2-b2e2-4f4b-a35f-8186d4472b82-manifests\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.781649 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.781487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0727af2-b2e2-4f4b-a35f-8186d4472b82-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.781649 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.781526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0727af2-b2e2-4f4b-a35f-8186d4472b82-tmp\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.781649 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.781600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0727af2-b2e2-4f4b-a35f-8186d4472b82-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.782081 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.781640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nghf\" (UniqueName: \"kubernetes.io/projected/f0727af2-b2e2-4f4b-a35f-8186d4472b82-kube-api-access-8nghf\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.883427 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.883374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0727af2-b2e2-4f4b-a35f-8186d4472b82-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.883603 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.883444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nghf\" (UniqueName: \"kubernetes.io/projected/f0727af2-b2e2-4f4b-a35f-8186d4472b82-kube-api-access-8nghf\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.883603 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.883493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0727af2-b2e2-4f4b-a35f-8186d4472b82-manifests\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.883603 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.883535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0727af2-b2e2-4f4b-a35f-8186d4472b82-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.883603 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.883561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0727af2-b2e2-4f4b-a35f-8186d4472b82-tmp\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.884114 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.884087    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0727af2-b2e2-4f4b-a35f-8186d4472b82-manifests\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.885998 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.885977    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0727af2-b2e2-4f4b-a35f-8186d4472b82-tmp\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.886256 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.886233    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0727af2-b2e2-4f4b-a35f-8186d4472b82-webhook-cert\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.886336 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.886313    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0727af2-b2e2-4f4b-a35f-8186d4472b82-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.897807 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.897780    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nghf\" (UniqueName: \"kubernetes.io/projected/f0727af2-b2e2-4f4b-a35f-8186d4472b82-kube-api-access-8nghf\") pod \"opendatahub-operator-controller-manager-76cd65fb98-qxjkr\" (UID: \"f0727af2-b2e2-4f4b-a35f-8186d4472b82\") " pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:20.992012 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:20.991935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:21.150179 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:21.150147    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"]
Jul 07 18:00:21.153306 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:21.153276    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0727af2_b2e2_4f4b_a35f_8186d4472b82.slice/crio-cbcfa60046e5b89db82c1d27e5c8583366182f86446345d8ca25bf6db7222411 WatchSource:0}: Error finding container cbcfa60046e5b89db82c1d27e5c8583366182f86446345d8ca25bf6db7222411: Status 404 returned error can't find the container with id cbcfa60046e5b89db82c1d27e5c8583366182f86446345d8ca25bf6db7222411
Jul 07 18:00:21.182618 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:21.182585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr" event={"ID":"f0727af2-b2e2-4f4b-a35f-8186d4472b82","Type":"ContainerStarted","Data":"cbcfa60046e5b89db82c1d27e5c8583366182f86446345d8ca25bf6db7222411"}
Jul 07 18:00:25.204409 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:25.204372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr" event={"ID":"f0727af2-b2e2-4f4b-a35f-8186d4472b82","Type":"ContainerStarted","Data":"fb98fc7909402866952276d866f114014d8380a2f64e93c75c69f79daa15a78c"}
Jul 07 18:00:26.210462 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:26.210425    2580 generic.go:356] "Generic (PLEG): container finished" podID="f0727af2-b2e2-4f4b-a35f-8186d4472b82" containerID="fb98fc7909402866952276d866f114014d8380a2f64e93c75c69f79daa15a78c" exitCode=0
Jul 07 18:00:26.210933 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:26.210514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr" event={"ID":"f0727af2-b2e2-4f4b-a35f-8186d4472b82","Type":"ContainerDied","Data":"fb98fc7909402866952276d866f114014d8380a2f64e93c75c69f79daa15a78c"}
Jul 07 18:00:27.218928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:27.218873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr" event={"ID":"f0727af2-b2e2-4f4b-a35f-8186d4472b82","Type":"ContainerStarted","Data":"e9fe88b2ba6a14cebe2897e686f752b1d2be9a9dbbc5e1c35c90fb8f7c8439e6"}
Jul 07 18:00:27.219395 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:27.219029    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:27.246137 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:27.246087    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr" podStartSLOduration=2.82779382 podStartE2EDuration="7.246070902s" totalImagesPullingTime="3.347309263s" totalInitContainerRuntime="1.070967819s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:27.244530291 +0000 UTC m=+389.134891016" watchObservedRunningTime="2026-07-07 18:00:27.246070902 +0000 UTC m=+389.136431626"
Jul 07 18:00:38.226175 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:38.226142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76cd65fb98-qxjkr"
Jul 07 18:00:42.524224 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.524191    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"]
Jul 07 18:00:42.535347 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.535316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.560422 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.560387    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 18:00:42.560589 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.560533    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 18:00:42.565390 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.565364    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wmjsb"]
Jul 07 18:00:42.567894 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.567867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec135987-68d8-4691-9177-675d3f6bf693-tls-certs\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.568073 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.567970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec135987-68d8-4691-9177-675d3f6bf693-tmp\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.568073 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.568017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlwwh\" (UniqueName: \"kubernetes.io/projected/ec135987-68d8-4691-9177-675d3f6bf693-kube-api-access-wlwwh\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.568292 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.568275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:42.571534 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.571512    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 18:00:42.573732 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.573715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dj2qx\""
Jul 07 18:00:42.574176 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.574149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 18:00:42.578263 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.578245    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 18:00:42.608688 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.608648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wmjsb"]
Jul 07 18:00:42.610335 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.610305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"]
Jul 07 18:00:42.668402 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.668369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec135987-68d8-4691-9177-675d3f6bf693-tmp\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.668588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.668419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-cert\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:42.668588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.668455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwwh\" (UniqueName: \"kubernetes.io/projected/ec135987-68d8-4691-9177-675d3f6bf693-kube-api-access-wlwwh\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.668588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.668505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm2d4\" (UniqueName: \"kubernetes.io/projected/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-kube-api-access-dm2d4\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:42.668588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.668547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec135987-68d8-4691-9177-675d3f6bf693-tls-certs\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.670837 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.670812    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec135987-68d8-4691-9177-675d3f6bf693-tmp\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.671097 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.671075    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec135987-68d8-4691-9177-675d3f6bf693-tls-certs\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.730325 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.730289    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlwwh\" (UniqueName: \"kubernetes.io/projected/ec135987-68d8-4691-9177-675d3f6bf693-kube-api-access-wlwwh\") pod \"kube-auth-proxy-74ffcdc9c9-w7z4z\" (UID: \"ec135987-68d8-4691-9177-675d3f6bf693\") " pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:42.769831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.769800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm2d4\" (UniqueName: \"kubernetes.io/projected/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-kube-api-access-dm2d4\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:42.770026 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.769858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-cert\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:42.770026 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:42.769960    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 18:00:42.770026 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:42.770014    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-cert podName:f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74 nodeName:}" failed. No retries permitted until 2026-07-07 18:00:43.26999721 +0000 UTC m=+405.160357913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-cert") pod "odh-model-controller-7998c67dc4-wmjsb" (UID: "f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74") : secret "odh-model-controller-webhook-cert" not found
Jul 07 18:00:42.824619 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.824534    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm2d4\" (UniqueName: \"kubernetes.io/projected/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-kube-api-access-dm2d4\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:42.845519 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:42.845482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"
Jul 07 18:00:43.073087 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:43.073051    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec135987_68d8_4691_9177_675d3f6bf693.slice/crio-36f3f3d70e0e1405dce44ca42aacc58bac4b90b915240cb97015113abc8311ca WatchSource:0}: Error finding container 36f3f3d70e0e1405dce44ca42aacc58bac4b90b915240cb97015113abc8311ca: Status 404 returned error can't find the container with id 36f3f3d70e0e1405dce44ca42aacc58bac4b90b915240cb97015113abc8311ca
Jul 07 18:00:43.093203 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:43.093175    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z"]
Jul 07 18:00:43.273463 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:43.273428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-cert\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:43.275882 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:43.275863    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74-cert\") pod \"odh-model-controller-7998c67dc4-wmjsb\" (UID: \"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74\") " pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:43.289588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:43.289552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z" event={"ID":"ec135987-68d8-4691-9177-675d3f6bf693","Type":"ContainerStarted","Data":"36f3f3d70e0e1405dce44ca42aacc58bac4b90b915240cb97015113abc8311ca"}
Jul 07 18:00:43.479161 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:43.479120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:43.685610 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:43.685579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f9825b_d98b_4fb0_a3c1_b0d95a4a6a74.slice/crio-705ae809af402bb41f7b269d6a9fcabb9f6ca2170a23bcddfa9358752fc55865 WatchSource:0}: Error finding container 705ae809af402bb41f7b269d6a9fcabb9f6ca2170a23bcddfa9358752fc55865: Status 404 returned error can't find the container with id 705ae809af402bb41f7b269d6a9fcabb9f6ca2170a23bcddfa9358752fc55865
Jul 07 18:00:43.687817 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:43.687791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wmjsb"]
Jul 07 18:00:44.295550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.295513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb" event={"ID":"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74","Type":"ContainerStarted","Data":"705ae809af402bb41f7b269d6a9fcabb9f6ca2170a23bcddfa9358752fc55865"}
Jul 07 18:00:44.540499 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.540441    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pm4lx"]
Jul 07 18:00:44.543307 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.543280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:44.550633 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.550564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 18:00:44.572925 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.572877    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pm4lx"]
Jul 07 18:00:44.585674 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.585620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:44.585846 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.585705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqgjm\" (UniqueName: \"kubernetes.io/projected/2dc34b09-2df4-409a-add8-bae0cfb178d8-kube-api-access-bqgjm\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:44.686600 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.686552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqgjm\" (UniqueName: \"kubernetes.io/projected/2dc34b09-2df4-409a-add8-bae0cfb178d8-kube-api-access-bqgjm\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:44.687080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.686693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:44.687080 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:44.686887    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 07 18:00:44.687080 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:44.686977    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs podName:2dc34b09-2df4-409a-add8-bae0cfb178d8 nodeName:}" failed. No retries permitted until 2026-07-07 18:00:45.186954638 +0000 UTC m=+407.077315361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs") pod "model-serving-api-54d89bd5fc-pm4lx" (UID: "2dc34b09-2df4-409a-add8-bae0cfb178d8") : secret "model-serving-api-tls" not found
Jul 07 18:00:44.705181 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:44.702778    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqgjm\" (UniqueName: \"kubernetes.io/projected/2dc34b09-2df4-409a-add8-bae0cfb178d8-kube-api-access-bqgjm\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:45.190861 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.190816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:45.191064 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:45.191018    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 07 18:00:45.191113 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:45.191082    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs podName:2dc34b09-2df4-409a-add8-bae0cfb178d8 nodeName:}" failed. No retries permitted until 2026-07-07 18:00:46.191067348 +0000 UTC m=+408.081428049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs") pod "model-serving-api-54d89bd5fc-pm4lx" (UID: "2dc34b09-2df4-409a-add8-bae0cfb178d8") : secret "model-serving-api-tls" not found
Jul 07 18:00:45.629345 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.628186    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"]
Jul 07 18:00:45.631928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.631886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.642701 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.642531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 07 18:00:45.642801 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.642748    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 07 18:00:45.643484 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.643451    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 18:00:45.643641 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.643500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 07 18:00:45.643641 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.643507    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gbnr2\""
Jul 07 18:00:45.666877 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.666842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"]
Jul 07 18:00:45.694959 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.694929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.695414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.694975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b43000-37c7-4541-a28e-81719fad5f97-local-certs\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.695414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.695015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.695414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.695040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8fst\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-kube-api-access-k8fst\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.695414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.695155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-cacerts\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.695414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.695194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.695414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.695310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-istio-token\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796250 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796453 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8fst\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-kube-api-access-k8fst\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796453 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-cacerts\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796453 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796453 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-istio-token\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796453 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.796719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.796484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b43000-37c7-4541-a28e-81719fad5f97-local-certs\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.797331 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.797269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.799803 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.799765    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b43000-37c7-4541-a28e-81719fad5f97-local-certs\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.800012 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.799956    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-cacerts\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.800128 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.800105    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.801399 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.801376    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.825588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.825548    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8fst\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-kube-api-access-k8fst\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.826211 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.826158    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-istio-token\") pod \"istiod-openshift-gateway-7574459656-8fdxj\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:45.946797 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:45.946707    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:46.200538 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:46.200465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:46.203830 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:46.203797    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dc34b09-2df4-409a-add8-bae0cfb178d8-tls-certs\") pod \"model-serving-api-54d89bd5fc-pm4lx\" (UID: \"2dc34b09-2df4-409a-add8-bae0cfb178d8\") " pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:46.359570 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:46.359532    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:47.175517 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:47.175489    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pm4lx"]
Jul 07 18:00:47.177672 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:47.177630    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc34b09_2df4_409a_add8_bae0cfb178d8.slice/crio-7deeb732274e5480360397fcba32cc5db43bfc55af49be65ff3afd066988f7d1 WatchSource:0}: Error finding container 7deeb732274e5480360397fcba32cc5db43bfc55af49be65ff3afd066988f7d1: Status 404 returned error can't find the container with id 7deeb732274e5480360397fcba32cc5db43bfc55af49be65ff3afd066988f7d1
Jul 07 18:00:47.181498 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:47.181472    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"]
Jul 07 18:00:47.182439 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:47.182405    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b43000_37c7_4541_a28e_81719fad5f97.slice/crio-0f2d64d1c0ea02dff3a95736919d56f7dba62fc20cc1834a11820a8b1aa92835 WatchSource:0}: Error finding container 0f2d64d1c0ea02dff3a95736919d56f7dba62fc20cc1834a11820a8b1aa92835: Status 404 returned error can't find the container with id 0f2d64d1c0ea02dff3a95736919d56f7dba62fc20cc1834a11820a8b1aa92835
Jul 07 18:00:47.313106 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:47.312970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx" event={"ID":"2dc34b09-2df4-409a-add8-bae0cfb178d8","Type":"ContainerStarted","Data":"7deeb732274e5480360397fcba32cc5db43bfc55af49be65ff3afd066988f7d1"}
Jul 07 18:00:47.314425 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:47.314396    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" event={"ID":"b6b43000-37c7-4541-a28e-81719fad5f97","Type":"ContainerStarted","Data":"0f2d64d1c0ea02dff3a95736919d56f7dba62fc20cc1834a11820a8b1aa92835"}
Jul 07 18:00:47.316236 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:47.316194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z" event={"ID":"ec135987-68d8-4691-9177-675d3f6bf693","Type":"ContainerStarted","Data":"de2d235509b91e2c9c1d490d8edf4e389d97da0a1f35c98de9963ac0aefb8e69"}
Jul 07 18:00:48.327531 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:48.327411    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb" event={"ID":"f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74","Type":"ContainerStarted","Data":"b3280be1f505e6441dafb7596c90e890ae1cbe0c5b9a10ecead70487a3e9c84d"}
Jul 07 18:00:48.328005 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:48.327977    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:00:48.361953 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:48.361872    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb" podStartSLOduration=2.250721165 podStartE2EDuration="6.361854111s" totalImagesPullingTime="4.111132946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:48.360426126 +0000 UTC m=+410.250786863" watchObservedRunningTime="2026-07-07 18:00:48.361854111 +0000 UTC m=+410.252214834"
Jul 07 18:00:48.362923 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:48.362863    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74ffcdc9c9-w7z4z" podStartSLOduration=2.455143674 podStartE2EDuration="6.362848995s" totalImagesPullingTime="3.907705321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:47.413929251 +0000 UTC m=+409.304289966" watchObservedRunningTime="2026-07-07 18:00:48.362848995 +0000 UTC m=+410.253209721"
Jul 07 18:00:50.371135 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:50.368417    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:00:50.371135 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:50.368759    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:00:51.343938 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.343887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx" event={"ID":"2dc34b09-2df4-409a-add8-bae0cfb178d8","Type":"ContainerStarted","Data":"3b863e35dbdd8d856fff76159e9cec40d84cb39756600e56633926e41359a8ac"}
Jul 07 18:00:51.344561 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.344536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:00:51.346459 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.346437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" event={"ID":"b6b43000-37c7-4541-a28e-81719fad5f97","Type":"ContainerStarted","Data":"be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e"}
Jul 07 18:00:51.347601 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.347582    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:51.349527 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.349495    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-8fdxj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 18:00:51.349622 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.349561    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" podUID="b6b43000-37c7-4541-a28e-81719fad5f97" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 18:00:51.398930 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.398860    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx" podStartSLOduration=4.209553679 podStartE2EDuration="7.398837271s" totalImagesPullingTime="3.189283592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:51.396257801 +0000 UTC m=+413.286618524" watchObservedRunningTime="2026-07-07 18:00:51.398837271 +0000 UTC m=+413.289197997"
Jul 07 18:00:51.489034 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:51.488970    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" podStartSLOduration=3.305843136 podStartE2EDuration="6.488953785s" totalImagesPullingTime="3.183110649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:00:51.487001112 +0000 UTC m=+413.377361837" watchObservedRunningTime="2026-07-07 18:00:51.488953785 +0000 UTC m=+413.379314509"
Jul 07 18:00:52.351731 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:52.351695    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:00:58.536234 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.536199    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8225m"]
Jul 07 18:00:58.540082 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.540054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.542867 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:58.542835    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kserve-webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" type="*v1.Secret"
Jul 07 18:00:58.543178 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:58.543154    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kserve-controller-manager-dockercfg-8b67l\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8b67l\"" type="*v1.Secret"
Jul 07 18:00:58.543327 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:58.543294    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kserve-controller-manager-d6f4b65b8-8225m\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="9c174176-3864-4d15-8845-8b1327107b23" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.582791 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.582757    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"]
Jul 07 18:00:58.585552 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.585535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.597848 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.597823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8225m"]
Jul 07 18:00:58.600603 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.600582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4htt4\""
Jul 07 18:00:58.600719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.600590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 07 18:00:58.615080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.614971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dpxf\" (UniqueName: \"kubernetes.io/projected/9c174176-3864-4d15-8845-8b1327107b23-kube-api-access-6dpxf\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.615220 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.615107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c174176-3864-4d15-8845-8b1327107b23-cert\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.681316 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.681277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"]
Jul 07 18:00:58.715981 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.715952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sccnv\" (UID: \"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.716183 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.715997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkgdb\" (UniqueName: \"kubernetes.io/projected/bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6-kube-api-access-bkgdb\") pod \"llmisvc-controller-manager-777c6bd7d5-sccnv\" (UID: \"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.716183 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.716109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c174176-3864-4d15-8845-8b1327107b23-cert\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.716296 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.716179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dpxf\" (UniqueName: \"kubernetes.io/projected/9c174176-3864-4d15-8845-8b1327107b23-kube-api-access-6dpxf\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.768846 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.768809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dpxf\" (UniqueName: \"kubernetes.io/projected/9c174176-3864-4d15-8845-8b1327107b23-kube-api-access-6dpxf\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:00:58.817378 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.817286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sccnv\" (UID: \"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.817378 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.817346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkgdb\" (UniqueName: \"kubernetes.io/projected/bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6-kube-api-access-bkgdb\") pod \"llmisvc-controller-manager-777c6bd7d5-sccnv\" (UID: \"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.819873 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.819847    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sccnv\" (UID: \"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.846126 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.846104    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkgdb\" (UniqueName: \"kubernetes.io/projected/bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6-kube-api-access-bkgdb\") pod \"llmisvc-controller-manager-777c6bd7d5-sccnv\" (UID: \"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:58.897509 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:58.897476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:00:59.062049 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:59.062016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"]
Jul 07 18:00:59.068153 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:00:59.068090    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbfe1bb2a_7f13_431f_966e_0d1d65c3ddf6.slice/crio-ae860d7f1cc16698f9d551ded38eb5ccf1e74e41afdc8344516bcbddc1790db1 WatchSource:0}: Error finding container ae860d7f1cc16698f9d551ded38eb5ccf1e74e41afdc8344516bcbddc1790db1: Status 404 returned error can't find the container with id ae860d7f1cc16698f9d551ded38eb5ccf1e74e41afdc8344516bcbddc1790db1
Jul 07 18:00:59.396645 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:59.396612    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv" event={"ID":"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6","Type":"ContainerStarted","Data":"ae860d7f1cc16698f9d551ded38eb5ccf1e74e41afdc8344516bcbddc1790db1"}
Jul 07 18:00:59.716996 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:59.716922    2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: failed to sync secret cache: timed out waiting for the condition
Jul 07 18:00:59.717357 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:00:59.717008    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c174176-3864-4d15-8845-8b1327107b23-cert podName:9c174176-3864-4d15-8845-8b1327107b23 nodeName:}" failed. No retries permitted until 2026-07-07 18:01:00.216990978 +0000 UTC m=+422.107351680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9c174176-3864-4d15-8845-8b1327107b23-cert") pod "kserve-controller-manager-d6f4b65b8-8225m" (UID: "9c174176-3864-4d15-8845-8b1327107b23") : failed to sync secret cache: timed out waiting for the condition
Jul 07 18:00:59.822709 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:59.822680    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-8b67l\""
Jul 07 18:00:59.976660 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:00:59.976560    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 07 18:01:00.229653 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.229569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c174176-3864-4d15-8845-8b1327107b23-cert\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:01:00.232075 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.232049    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c174176-3864-4d15-8845-8b1327107b23-cert\") pod \"kserve-controller-manager-d6f4b65b8-8225m\" (UID: \"9c174176-3864-4d15-8845-8b1327107b23\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:01:00.340573 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.340538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wmjsb"
Jul 07 18:01:00.354123 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.354096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:01:00.556427 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.556398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8225m"]
Jul 07 18:01:00.558660 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:01:00.558630    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c174176_3864_4d15_8845_8b1327107b23.slice/crio-670b456742e1cebf1370f83a043e9801304661d6fc3cabaad7b868796a497222 WatchSource:0}: Error finding container 670b456742e1cebf1370f83a043e9801304661d6fc3cabaad7b868796a497222: Status 404 returned error can't find the container with id 670b456742e1cebf1370f83a043e9801304661d6fc3cabaad7b868796a497222
Jul 07 18:01:00.641127 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.641087    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"]
Jul 07 18:01:00.641346 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.641310    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" podUID="b6b43000-37c7-4541-a28e-81719fad5f97" containerName="discovery" containerID="cri-o://be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e" gracePeriod=30
Jul 07 18:01:00.886699 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.886677    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:01:00.935718 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-dns-cert\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.935718 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-ca-configmap\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.936027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-cacerts\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.936027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935807    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b43000-37c7-4541-a28e-81719fad5f97-local-certs\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.936027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-istio-token\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.936027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935868    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-kubeconfig\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.936027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.935932    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8fst\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-kube-api-access-k8fst\") pod \"b6b43000-37c7-4541-a28e-81719fad5f97\" (UID: \"b6b43000-37c7-4541-a28e-81719fad5f97\") "
Jul 07 18:01:00.936292 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.936177    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 18:01:00.938983 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.938885    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:01:00.939103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.938981    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:01:00.939103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.938986    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6b43000-37c7-4541-a28e-81719fad5f97-local-certs" (OuterVolumeSpecName: "local-certs") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:01:00.939103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.938993    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-kube-api-access-k8fst" (OuterVolumeSpecName: "kube-api-access-k8fst") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "kube-api-access-k8fst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:01:00.939103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.939007    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-cacerts" (OuterVolumeSpecName: "cacerts") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:01:00.939103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:00.939040    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b6b43000-37c7-4541-a28e-81719fad5f97" (UID: "b6b43000-37c7-4541-a28e-81719fad5f97"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:01:01.036763 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036732    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-dns-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.036763 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036761    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b43000-37c7-4541-a28e-81719fad5f97-istio-csr-ca-configmap\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.036992 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036772    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-cacerts\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.036992 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036783    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6b43000-37c7-4541-a28e-81719fad5f97-local-certs\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.036992 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036791    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-istio-token\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.036992 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036798    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6b43000-37c7-4541-a28e-81719fad5f97-istio-kubeconfig\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.036992 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.036805    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8fst\" (UniqueName: \"kubernetes.io/projected/b6b43000-37c7-4541-a28e-81719fad5f97-kube-api-access-k8fst\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:01.412278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.412242    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6b43000-37c7-4541-a28e-81719fad5f97" containerID="be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e" exitCode=0
Jul 07 18:01:01.412483 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.412292    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"
Jul 07 18:01:01.412483 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.412322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" event={"ID":"b6b43000-37c7-4541-a28e-81719fad5f97","Type":"ContainerDied","Data":"be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e"}
Jul 07 18:01:01.412483 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.412369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj" event={"ID":"b6b43000-37c7-4541-a28e-81719fad5f97","Type":"ContainerDied","Data":"0f2d64d1c0ea02dff3a95736919d56f7dba62fc20cc1834a11820a8b1aa92835"}
Jul 07 18:01:01.412483 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.412394    2580 scope.go:117] "RemoveContainer" containerID="be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e"
Jul 07 18:01:01.413835 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.413690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m" event={"ID":"9c174176-3864-4d15-8845-8b1327107b23","Type":"ContainerStarted","Data":"670b456742e1cebf1370f83a043e9801304661d6fc3cabaad7b868796a497222"}
Jul 07 18:01:01.422171 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.422142    2580 scope.go:117] "RemoveContainer" containerID="be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e"
Jul 07 18:01:01.422465 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:01:01.422446    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e\": container with ID starting with be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e not found: ID does not exist" containerID="be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e"
Jul 07 18:01:01.422534 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.422478    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e"} err="failed to get container status \"be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e\": rpc error: code = NotFound desc = could not find container \"be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e\": container with ID starting with be3389a9a7f228159de2e3787feaa8d499a5de194faceda471e34a4af503036e not found: ID does not exist"
Jul 07 18:01:01.451305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.451272    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"]
Jul 07 18:01:01.469294 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:01.469262    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8fdxj"]
Jul 07 18:01:02.356066 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:02.356031    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-pm4lx"
Jul 07 18:01:02.598884 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:02.598635    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6b43000-37c7-4541-a28e-81719fad5f97" path="/var/lib/kubelet/pods/b6b43000-37c7-4541-a28e-81719fad5f97/volumes"
Jul 07 18:01:03.426694 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:03.426607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m" event={"ID":"9c174176-3864-4d15-8845-8b1327107b23","Type":"ContainerStarted","Data":"e378ef7636aa8280767b5970084dcb45039c2e315bc89ab0f723448a84fbb873"}
Jul 07 18:01:03.427133 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:03.426695    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:01:03.428117 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:03.428097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv" event={"ID":"bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6","Type":"ContainerStarted","Data":"9e406f9b1d8ee8752b117d6337acc2d95b6ee5ff7ba94e908f21264437349720"}
Jul 07 18:01:03.428222 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:03.428204    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:01:03.495398 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:03.495343    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m" podStartSLOduration=2.920302978 podStartE2EDuration="5.495327325s" totalImagesPullingTime="2.575024347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:03.494148634 +0000 UTC m=+425.384509371" watchObservedRunningTime="2026-07-07 18:01:03.495327325 +0000 UTC m=+425.385688049"
Jul 07 18:01:03.567541 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:03.567493    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv" podStartSLOduration=2.2057059199999998 podStartE2EDuration="5.56747566s" totalImagesPullingTime="3.36176974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:03.564330913 +0000 UTC m=+425.454691638" watchObservedRunningTime="2026-07-07 18:01:03.56747566 +0000 UTC m=+425.457836397"
Jul 07 18:01:34.434763 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:34.434712    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sccnv"
Jul 07 18:01:34.438035 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:34.438011    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8225m"
Jul 07 18:01:43.210020 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.209981    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6mdfg"]
Jul 07 18:01:43.210628 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.210452    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6b43000-37c7-4541-a28e-81719fad5f97" containerName="discovery"
Jul 07 18:01:43.210628 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.210472    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6b43000-37c7-4541-a28e-81719fad5f97" containerName="discovery"
Jul 07 18:01:43.210628 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.210581    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6b43000-37c7-4541-a28e-81719fad5f97" containerName="discovery"
Jul 07 18:01:43.213647 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.213622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:43.216618 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.216592    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 18:01:43.217444 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.217427    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 18:01:43.217668 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.217653    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-prqrk\""
Jul 07 18:01:43.230398 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.230364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6mdfg"]
Jul 07 18:01:43.297729 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.297692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z782\" (UniqueName: \"kubernetes.io/projected/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d-kube-api-access-2z782\") pod \"kuadrant-operator-catalog-6mdfg\" (UID: \"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d\") " pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:43.398314 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.398273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z782\" (UniqueName: \"kubernetes.io/projected/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d-kube-api-access-2z782\") pod \"kuadrant-operator-catalog-6mdfg\" (UID: \"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d\") " pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:43.408795 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.408755    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z782\" (UniqueName: \"kubernetes.io/projected/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d-kube-api-access-2z782\") pod \"kuadrant-operator-catalog-6mdfg\" (UID: \"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d\") " pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:43.501128 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.501041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6mdfg"]
Jul 07 18:01:43.501304 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.501292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:43.649988 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:01:43.649957    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40cd2ef6_0d87_4ffb_a10c_8eab5da0f94d.slice/crio-6824f72a7ad76813d60d4fbd9049961ccfb1fdbc12d0f8a54a5cb771a3d583e4 WatchSource:0}: Error finding container 6824f72a7ad76813d60d4fbd9049961ccfb1fdbc12d0f8a54a5cb771a3d583e4: Status 404 returned error can't find the container with id 6824f72a7ad76813d60d4fbd9049961ccfb1fdbc12d0f8a54a5cb771a3d583e4
Jul 07 18:01:43.652476 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.652447    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6mdfg"]
Jul 07 18:01:43.727760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.727728    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ct58d"]
Jul 07 18:01:43.730695 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.730678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:43.743862 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.743837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ct58d"]
Jul 07 18:01:43.801286 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.801194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxpq2\" (UniqueName: \"kubernetes.io/projected/ada81ba7-7c02-4d96-88a7-7d1c3527e425-kube-api-access-kxpq2\") pod \"kuadrant-operator-catalog-ct58d\" (UID: \"ada81ba7-7c02-4d96-88a7-7d1c3527e425\") " pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:43.902744 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.902701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxpq2\" (UniqueName: \"kubernetes.io/projected/ada81ba7-7c02-4d96-88a7-7d1c3527e425-kube-api-access-kxpq2\") pod \"kuadrant-operator-catalog-ct58d\" (UID: \"ada81ba7-7c02-4d96-88a7-7d1c3527e425\") " pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:43.914292 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:43.914257    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxpq2\" (UniqueName: \"kubernetes.io/projected/ada81ba7-7c02-4d96-88a7-7d1c3527e425-kube-api-access-kxpq2\") pod \"kuadrant-operator-catalog-ct58d\" (UID: \"ada81ba7-7c02-4d96-88a7-7d1c3527e425\") " pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:44.041038 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:44.040990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:44.222720 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:44.222684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ct58d"]
Jul 07 18:01:44.228518 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:01:44.228491    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada81ba7_7c02_4d96_88a7_7d1c3527e425.slice/crio-39774d6ee62d5402261c06e60e2878e2ae3c647132914bdc8f9e97ab2859a7a3 WatchSource:0}: Error finding container 39774d6ee62d5402261c06e60e2878e2ae3c647132914bdc8f9e97ab2859a7a3: Status 404 returned error can't find the container with id 39774d6ee62d5402261c06e60e2878e2ae3c647132914bdc8f9e97ab2859a7a3
Jul 07 18:01:44.619598 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:44.619561    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg" event={"ID":"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d","Type":"ContainerStarted","Data":"6824f72a7ad76813d60d4fbd9049961ccfb1fdbc12d0f8a54a5cb771a3d583e4"}
Jul 07 18:01:44.620698 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:44.620673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ct58d" event={"ID":"ada81ba7-7c02-4d96-88a7-7d1c3527e425","Type":"ContainerStarted","Data":"39774d6ee62d5402261c06e60e2878e2ae3c647132914bdc8f9e97ab2859a7a3"}
Jul 07 18:01:46.634874 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.634837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ct58d" event={"ID":"ada81ba7-7c02-4d96-88a7-7d1c3527e425","Type":"ContainerStarted","Data":"c1177ef0f8a509916d86550ac3b41cfd395a1a1096be936683775761bad1b29a"}
Jul 07 18:01:46.636388 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.636365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg" event={"ID":"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d","Type":"ContainerStarted","Data":"462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e"}
Jul 07 18:01:46.636530 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.636473    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg" podUID="40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" containerName="registry-server" containerID="cri-o://462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e" gracePeriod=2
Jul 07 18:01:46.656303 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.656244    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ct58d" podStartSLOduration=2.130168808 podStartE2EDuration="3.65622653s" totalImagesPullingTime="1.526057722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:46.655082853 +0000 UTC m=+468.545443580" watchObservedRunningTime="2026-07-07 18:01:46.65622653 +0000 UTC m=+468.546587284"
Jul 07 18:01:46.674135 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.674058    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg" podStartSLOduration=1.569734946 podStartE2EDuration="3.674036063s" totalImagesPullingTime="2.104301117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:01:46.673520047 +0000 UTC m=+468.563880772" watchObservedRunningTime="2026-07-07 18:01:46.674036063 +0000 UTC m=+468.564396788"
Jul 07 18:01:46.885537 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.885477    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:46.930818 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.930784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z782\" (UniqueName: \"kubernetes.io/projected/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d-kube-api-access-2z782\") pod \"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d\" (UID: \"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d\") "
Jul 07 18:01:46.933157 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:46.933127    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d-kube-api-access-2z782" (OuterVolumeSpecName: "kube-api-access-2z782") pod "40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" (UID: "40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d"). InnerVolumeSpecName "kube-api-access-2z782". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:01:47.031867 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.031829    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z782\" (UniqueName: \"kubernetes.io/projected/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d-kube-api-access-2z782\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:01:47.642287 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.642250    2580 generic.go:356] "Generic (PLEG): container finished" podID="40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" containerID="462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e" exitCode=0
Jul 07 18:01:47.642742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.642295    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg"
Jul 07 18:01:47.642742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.642338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg" event={"ID":"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d","Type":"ContainerDied","Data":"462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e"}
Jul 07 18:01:47.642742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.642392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6mdfg" event={"ID":"40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d","Type":"ContainerDied","Data":"6824f72a7ad76813d60d4fbd9049961ccfb1fdbc12d0f8a54a5cb771a3d583e4"}
Jul 07 18:01:47.642742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.642412    2580 scope.go:117] "RemoveContainer" containerID="462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e"
Jul 07 18:01:47.652242 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.652223    2580 scope.go:117] "RemoveContainer" containerID="462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e"
Jul 07 18:01:47.652512 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:01:47.652495    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e\": container with ID starting with 462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e not found: ID does not exist" containerID="462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e"
Jul 07 18:01:47.652561 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.652522    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e"} err="failed to get container status \"462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e\": rpc error: code = NotFound desc = could not find container \"462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e\": container with ID starting with 462297208b1d46bd0dafe5195783520ff2b36451f990c60eef4a6ffe89f9f32e not found: ID does not exist"
Jul 07 18:01:47.665217 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.665192    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6mdfg"]
Jul 07 18:01:47.672363 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:47.672335    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6mdfg"]
Jul 07 18:01:48.595250 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:48.595212    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" path="/var/lib/kubelet/pods/40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d/volumes"
Jul 07 18:01:54.041588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:54.041544    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:54.042053 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:54.041623    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:54.064793 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:54.064766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:54.699145 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:54.699118    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ct58d"
Jul 07 18:01:58.559864 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.559828    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"]
Jul 07 18:01:58.560244 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.560228    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" containerName="registry-server"
Jul 07 18:01:58.560244 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.560240    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" containerName="registry-server"
Jul 07 18:01:58.560328 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.560300    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="40cd2ef6-0d87-4ffb-a10c-8eab5da0f94d" containerName="registry-server"
Jul 07 18:01:58.563091 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.563070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.565946 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.565922    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xxlhd\""
Jul 07 18:01:58.574549 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.574526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"]
Jul 07 18:01:58.623334 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.623281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.623334 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.623342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.623558 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.623366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ptl\" (UniqueName: \"kubernetes.io/projected/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-kube-api-access-r9ptl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.724641 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.724602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.724829 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.724653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.724829 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.724687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ptl\" (UniqueName: \"kubernetes.io/projected/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-kube-api-access-r9ptl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.725125 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.725102    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.725178 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.725129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.739042 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.739006    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ptl\" (UniqueName: \"kubernetes.io/projected/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-kube-api-access-r9ptl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:58.872170 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:58.872140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:01:59.014021 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.013992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"]
Jul 07 18:01:59.016347 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:01:59.016322    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd9a90e_c5fc_46ee_92db_5c5a73cbe7d0.slice/crio-a06b6c89eed1f231ae453350ede1a7fe212c8ce32fb0ba6974682a5a3af3b482 WatchSource:0}: Error finding container a06b6c89eed1f231ae453350ede1a7fe212c8ce32fb0ba6974682a5a3af3b482: Status 404 returned error can't find the container with id a06b6c89eed1f231ae453350ede1a7fe212c8ce32fb0ba6974682a5a3af3b482
Jul 07 18:01:59.358616 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.358584    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"]
Jul 07 18:01:59.360883 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.360867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.375319 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.375249    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"]
Jul 07 18:01:59.431408 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.431369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.431408 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.431413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ltw8\" (UniqueName: \"kubernetes.io/projected/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-kube-api-access-2ltw8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.431656 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.431504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.532133 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.532098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.532133 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.532134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ltw8\" (UniqueName: \"kubernetes.io/projected/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-kube-api-access-2ltw8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.532364 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.532185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.532530 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.532512    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.532579 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.532527    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.543225 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.543198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ltw8\" (UniqueName: \"kubernetes.io/projected/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-kube-api-access-2ltw8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.670350 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.670261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:01:59.704151 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.704043    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerID="1748e9a136af46e75074d6539e7187a7e65ceec43f917ced3ddb1f5e6bc4aaf9" exitCode=0
Jul 07 18:01:59.704350 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.704179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27" event={"ID":"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0","Type":"ContainerDied","Data":"1748e9a136af46e75074d6539e7187a7e65ceec43f917ced3ddb1f5e6bc4aaf9"}
Jul 07 18:01:59.704350 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.704228    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27" event={"ID":"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0","Type":"ContainerStarted","Data":"a06b6c89eed1f231ae453350ede1a7fe212c8ce32fb0ba6974682a5a3af3b482"}
Jul 07 18:01:59.811221 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.811195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"]
Jul 07 18:01:59.812939 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:01:59.812893    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318a0eb3_c407_4bae_9cd3_5ac4d511ced8.slice/crio-89e9e1c8f3b23c52c3884a16bb9c88f1920ed2c19f1f5d19f23b6bcbb8aad617 WatchSource:0}: Error finding container 89e9e1c8f3b23c52c3884a16bb9c88f1920ed2c19f1f5d19f23b6bcbb8aad617: Status 404 returned error can't find the container with id 89e9e1c8f3b23c52c3884a16bb9c88f1920ed2c19f1f5d19f23b6bcbb8aad617
Jul 07 18:01:59.968923 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.968881    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"]
Jul 07 18:01:59.971290 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.971274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:01:59.985699 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:01:59.985669    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"]
Jul 07 18:02:00.036865 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.036829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.037103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.036950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlvg\" (UniqueName: \"kubernetes.io/projected/8c4a69c2-9565-4d59-9b5b-396b599671b4-kube-api-access-5rlvg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.037103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.037007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.138030 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.137994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.138030 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.138048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlvg\" (UniqueName: \"kubernetes.io/projected/8c4a69c2-9565-4d59-9b5b-396b599671b4-kube-api-access-5rlvg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.138276 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.138091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.139015 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.138925    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.139015 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.138968    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.150276 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.150246    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlvg\" (UniqueName: \"kubernetes.io/projected/8c4a69c2-9565-4d59-9b5b-396b599671b4-kube-api-access-5rlvg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.298265 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.298186    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:00.365533 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.365498    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"]
Jul 07 18:02:00.368865 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.368831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.383238 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.383192    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"]
Jul 07 18:02:00.440403 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.440362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z92g\" (UniqueName: \"kubernetes.io/projected/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-kube-api-access-9z92g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.440576 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.440431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.440576 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.440459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.455300 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.455277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"]
Jul 07 18:02:00.457504 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:00.457471    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4a69c2_9565_4d59_9b5b_396b599671b4.slice/crio-dadba65b109b399102743584a0c9345f8b761a71f006cb135d797d98620481bb WatchSource:0}: Error finding container dadba65b109b399102743584a0c9345f8b761a71f006cb135d797d98620481bb: Status 404 returned error can't find the container with id dadba65b109b399102743584a0c9345f8b761a71f006cb135d797d98620481bb
Jul 07 18:02:00.541632 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.541580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.541815 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.541662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.541815 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.541760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z92g\" (UniqueName: \"kubernetes.io/projected/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-kube-api-access-9z92g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.542034 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.542016    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.542078 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.542031    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.553399 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.553326    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z92g\" (UniqueName: \"kubernetes.io/projected/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-kube-api-access-9z92g\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.681005 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.680967    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:00.711545 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.711508    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerID="b6d5e288ea6a22c0d1ea4643d1fd3e4c7c35a162bb271ffbb63bf5e9d9ec1cfe" exitCode=0
Jul 07 18:02:00.711545 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.711538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn" event={"ID":"8c4a69c2-9565-4d59-9b5b-396b599671b4","Type":"ContainerDied","Data":"b6d5e288ea6a22c0d1ea4643d1fd3e4c7c35a162bb271ffbb63bf5e9d9ec1cfe"}
Jul 07 18:02:00.711791 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.711586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn" event={"ID":"8c4a69c2-9565-4d59-9b5b-396b599671b4","Type":"ContainerStarted","Data":"dadba65b109b399102743584a0c9345f8b761a71f006cb135d797d98620481bb"}
Jul 07 18:02:00.713784 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.713757    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerID="74f8a2c4a88b329e0dd24fc5d595a1fd48aec4c27626c34e97b55a36c782a640" exitCode=0
Jul 07 18:02:00.713941 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.713842    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27" event={"ID":"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0","Type":"ContainerDied","Data":"74f8a2c4a88b329e0dd24fc5d595a1fd48aec4c27626c34e97b55a36c782a640"}
Jul 07 18:02:00.715632 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.715520    2580 generic.go:356] "Generic (PLEG): container finished" podID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerID="1a9947358be3818d622bed55a42125fc143273200980b6fd534fc6f39e7b08a0" exitCode=0
Jul 07 18:02:00.715632 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.715570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj" event={"ID":"318a0eb3-c407-4bae-9cd3-5ac4d511ced8","Type":"ContainerDied","Data":"1a9947358be3818d622bed55a42125fc143273200980b6fd534fc6f39e7b08a0"}
Jul 07 18:02:00.715632 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.715606    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj" event={"ID":"318a0eb3-c407-4bae-9cd3-5ac4d511ced8","Type":"ContainerStarted","Data":"89e9e1c8f3b23c52c3884a16bb9c88f1920ed2c19f1f5d19f23b6bcbb8aad617"}
Jul 07 18:02:00.835111 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:00.835077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"]
Jul 07 18:02:00.836696 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:00.836671    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffec9a6_dc28_4fbf_9fdb_75bcdd5e819f.slice/crio-9ac282f4a817cdec49838b84cebc2a1707f2b6623e03434f84b322ea55445f4c WatchSource:0}: Error finding container 9ac282f4a817cdec49838b84cebc2a1707f2b6623e03434f84b322ea55445f4c: Status 404 returned error can't find the container with id 9ac282f4a817cdec49838b84cebc2a1707f2b6623e03434f84b322ea55445f4c
Jul 07 18:02:01.723432 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.723400    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerID="4b272e23f7dc66994b90f6fa8620e2a880773fdc01bd08f78086a6009561ece7" exitCode=0
Jul 07 18:02:01.723876 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.723456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss" event={"ID":"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f","Type":"ContainerDied","Data":"4b272e23f7dc66994b90f6fa8620e2a880773fdc01bd08f78086a6009561ece7"}
Jul 07 18:02:01.723876 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.723511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss" event={"ID":"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f","Type":"ContainerStarted","Data":"9ac282f4a817cdec49838b84cebc2a1707f2b6623e03434f84b322ea55445f4c"}
Jul 07 18:02:01.726059 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.726036    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerID="d4f3ae8cd19a5ef689a8a2de8d716fc8a0d43e7a8d29c705de4b3d4ce355fd98" exitCode=0
Jul 07 18:02:01.726151 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.726127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27" event={"ID":"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0","Type":"ContainerDied","Data":"d4f3ae8cd19a5ef689a8a2de8d716fc8a0d43e7a8d29c705de4b3d4ce355fd98"}
Jul 07 18:02:01.728022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.728000    2580 generic.go:356] "Generic (PLEG): container finished" podID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerID="44864c0741562c919b268762cfcdc0f350f83c34cedfc527c201b91a0aef105e" exitCode=0
Jul 07 18:02:01.728124 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.728088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj" event={"ID":"318a0eb3-c407-4bae-9cd3-5ac4d511ced8","Type":"ContainerDied","Data":"44864c0741562c919b268762cfcdc0f350f83c34cedfc527c201b91a0aef105e"}
Jul 07 18:02:01.730013 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.729993    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerID="a3287c316fc50480c374bf898e7d15e5747ea833fb310d069d888b1fc32d02d4" exitCode=0
Jul 07 18:02:01.730091 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:01.730025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn" event={"ID":"8c4a69c2-9565-4d59-9b5b-396b599671b4","Type":"ContainerDied","Data":"a3287c316fc50480c374bf898e7d15e5747ea833fb310d069d888b1fc32d02d4"}
Jul 07 18:02:02.736954 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.736899    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerID="78d86fd75455a4b826afee22fe345a0032b175698d193051e6a3c76f191ae00f" exitCode=0
Jul 07 18:02:02.737387 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.736968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn" event={"ID":"8c4a69c2-9565-4d59-9b5b-396b599671b4","Type":"ContainerDied","Data":"78d86fd75455a4b826afee22fe345a0032b175698d193051e6a3c76f191ae00f"}
Jul 07 18:02:02.738780 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.738756    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerID="eb42a401bf298509c558d991ce69a35d9216b3abed23ffb623f396def22cce1f" exitCode=0
Jul 07 18:02:02.738939 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.738818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss" event={"ID":"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f","Type":"ContainerDied","Data":"eb42a401bf298509c558d991ce69a35d9216b3abed23ffb623f396def22cce1f"}
Jul 07 18:02:02.741054 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.741034    2580 generic.go:356] "Generic (PLEG): container finished" podID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerID="4949f0460c034030c982403682c957bfa52ee729b76b2f74ff8fee666e4f09fa" exitCode=0
Jul 07 18:02:02.741138 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.741104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj" event={"ID":"318a0eb3-c407-4bae-9cd3-5ac4d511ced8","Type":"ContainerDied","Data":"4949f0460c034030c982403682c957bfa52ee729b76b2f74ff8fee666e4f09fa"}
Jul 07 18:02:02.888155 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.888128    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:02:02.964445 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.964413    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9ptl\" (UniqueName: \"kubernetes.io/projected/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-kube-api-access-r9ptl\") pod \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") "
Jul 07 18:02:02.964601 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.964493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-bundle\") pod \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") "
Jul 07 18:02:02.964601 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.964511    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-util\") pod \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\" (UID: \"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0\") "
Jul 07 18:02:02.965134 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.965105    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-bundle" (OuterVolumeSpecName: "bundle") pod "ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" (UID: "ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:02.966559 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.966534    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-kube-api-access-r9ptl" (OuterVolumeSpecName: "kube-api-access-r9ptl") pod "ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" (UID: "ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0"). InnerVolumeSpecName "kube-api-access-r9ptl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:02.970096 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:02.970075    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-util" (OuterVolumeSpecName: "util") pod "ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" (UID: "ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:03.065714 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.065635    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9ptl\" (UniqueName: \"kubernetes.io/projected/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-kube-api-access-r9ptl\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:03.065714 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.065664    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:03.065714 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.065674    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:03.749344 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.749303    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerID="91e9ac53a9b86a655ff5ae86d25bbbcf02996303bfcbc17ae83d1a9145d48566" exitCode=0
Jul 07 18:02:03.749749 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.749386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss" event={"ID":"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f","Type":"ContainerDied","Data":"91e9ac53a9b86a655ff5ae86d25bbbcf02996303bfcbc17ae83d1a9145d48566"}
Jul 07 18:02:03.751279 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.751262    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27"
Jul 07 18:02:03.751398 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.751296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27" event={"ID":"ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0","Type":"ContainerDied","Data":"a06b6c89eed1f231ae453350ede1a7fe212c8ce32fb0ba6974682a5a3af3b482"}
Jul 07 18:02:03.751398 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.751328    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a06b6c89eed1f231ae453350ede1a7fe212c8ce32fb0ba6974682a5a3af3b482"
Jul 07 18:02:03.909659 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.909624    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:03.911657 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.911637    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:02:03.973461 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.973428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-util\") pod \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") "
Jul 07 18:02:03.973631 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.973478    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-bundle\") pod \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") "
Jul 07 18:02:03.973631 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.973510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rlvg\" (UniqueName: \"kubernetes.io/projected/8c4a69c2-9565-4d59-9b5b-396b599671b4-kube-api-access-5rlvg\") pod \"8c4a69c2-9565-4d59-9b5b-396b599671b4\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") "
Jul 07 18:02:03.973631 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.973548    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-bundle\") pod \"8c4a69c2-9565-4d59-9b5b-396b599671b4\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") "
Jul 07 18:02:03.973631 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.973583    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-util\") pod \"8c4a69c2-9565-4d59-9b5b-396b599671b4\" (UID: \"8c4a69c2-9565-4d59-9b5b-396b599671b4\") "
Jul 07 18:02:03.973847 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.973821    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ltw8\" (UniqueName: \"kubernetes.io/projected/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-kube-api-access-2ltw8\") pod \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\" (UID: \"318a0eb3-c407-4bae-9cd3-5ac4d511ced8\") "
Jul 07 18:02:03.974187 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.974143    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-bundle" (OuterVolumeSpecName: "bundle") pod "8c4a69c2-9565-4d59-9b5b-396b599671b4" (UID: "8c4a69c2-9565-4d59-9b5b-396b599671b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:03.974334 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.974306    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-bundle" (OuterVolumeSpecName: "bundle") pod "318a0eb3-c407-4bae-9cd3-5ac4d511ced8" (UID: "318a0eb3-c407-4bae-9cd3-5ac4d511ced8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:03.976161 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.976134    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-kube-api-access-2ltw8" (OuterVolumeSpecName: "kube-api-access-2ltw8") pod "318a0eb3-c407-4bae-9cd3-5ac4d511ced8" (UID: "318a0eb3-c407-4bae-9cd3-5ac4d511ced8"). InnerVolumeSpecName "kube-api-access-2ltw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:03.976428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.976410    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4a69c2-9565-4d59-9b5b-396b599671b4-kube-api-access-5rlvg" (OuterVolumeSpecName: "kube-api-access-5rlvg") pod "8c4a69c2-9565-4d59-9b5b-396b599671b4" (UID: "8c4a69c2-9565-4d59-9b5b-396b599671b4"). InnerVolumeSpecName "kube-api-access-5rlvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:03.979875 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.979837    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-util" (OuterVolumeSpecName: "util") pod "8c4a69c2-9565-4d59-9b5b-396b599671b4" (UID: "8c4a69c2-9565-4d59-9b5b-396b599671b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:03.980001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:03.979841    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-util" (OuterVolumeSpecName: "util") pod "318a0eb3-c407-4bae-9cd3-5ac4d511ced8" (UID: "318a0eb3-c407-4bae-9cd3-5ac4d511ced8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:04.075278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.075194    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:04.075278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.075224    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ltw8\" (UniqueName: \"kubernetes.io/projected/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-kube-api-access-2ltw8\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:04.075278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.075235    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:04.075278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.075243    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/318a0eb3-c407-4bae-9cd3-5ac4d511ced8-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:04.075278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.075251    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rlvg\" (UniqueName: \"kubernetes.io/projected/8c4a69c2-9565-4d59-9b5b-396b599671b4-kube-api-access-5rlvg\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:04.075278 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.075259    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c4a69c2-9565-4d59-9b5b-396b599671b4-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:04.758742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.758711    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn"
Jul 07 18:02:04.759243 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.758742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn" event={"ID":"8c4a69c2-9565-4d59-9b5b-396b599671b4","Type":"ContainerDied","Data":"dadba65b109b399102743584a0c9345f8b761a71f006cb135d797d98620481bb"}
Jul 07 18:02:04.759243 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.758789    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dadba65b109b399102743584a0c9345f8b761a71f006cb135d797d98620481bb"
Jul 07 18:02:04.760650 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.760590    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj"
Jul 07 18:02:04.760650 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.760620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj" event={"ID":"318a0eb3-c407-4bae-9cd3-5ac4d511ced8","Type":"ContainerDied","Data":"89e9e1c8f3b23c52c3884a16bb9c88f1920ed2c19f1f5d19f23b6bcbb8aad617"}
Jul 07 18:02:04.760856 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.760655    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89e9e1c8f3b23c52c3884a16bb9c88f1920ed2c19f1f5d19f23b6bcbb8aad617"
Jul 07 18:02:04.897185 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.897150    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:04.982225 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.982193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-util\") pod \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") "
Jul 07 18:02:04.982399 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.982290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-bundle\") pod \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") "
Jul 07 18:02:04.982399 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.982321    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z92g\" (UniqueName: \"kubernetes.io/projected/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-kube-api-access-9z92g\") pod \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\" (UID: \"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f\") "
Jul 07 18:02:04.983044 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.983010    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-bundle" (OuterVolumeSpecName: "bundle") pod "5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" (UID: "5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:04.984733 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.984707    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-kube-api-access-9z92g" (OuterVolumeSpecName: "kube-api-access-9z92g") pod "5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" (UID: "5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f"). InnerVolumeSpecName "kube-api-access-9z92g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:04.989085 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:04.989062    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-util" (OuterVolumeSpecName: "util") pod "5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" (UID: "5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:05.083394 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:05.083341    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-util\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:05.083394 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:05.083387    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:05.083394 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:05.083398    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z92g\" (UniqueName: \"kubernetes.io/projected/5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f-kube-api-access-9z92g\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:05.767876 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:05.767844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss" event={"ID":"5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f","Type":"ContainerDied","Data":"9ac282f4a817cdec49838b84cebc2a1707f2b6623e03434f84b322ea55445f4c"}
Jul 07 18:02:05.768253 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:05.767889    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss"
Jul 07 18:02:05.768253 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:05.767892    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ac282f4a817cdec49838b84cebc2a1707f2b6623e03434f84b322ea55445f4c"
Jul 07 18:02:16.008412 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008373    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"]
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008699    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008711    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008717    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008722    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008731    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008737    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008743    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008748    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008762    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008773    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008778    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008798    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008803    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="util"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008810    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008815    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008821    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008826    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008831    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008836    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008846    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008851    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="pull"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008858    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="extract"
Jul 07 18:02:16.008866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.008863    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="extract"
Jul 07 18:02:16.009730 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.009011    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0" containerName="extract"
Jul 07 18:02:16.009730 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.009023    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="318a0eb3-c407-4bae-9cd3-5ac4d511ced8" containerName="extract"
Jul 07 18:02:16.009730 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.009036    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f" containerName="extract"
Jul 07 18:02:16.009730 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.009046    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c4a69c2-9565-4d59-9b5b-396b599671b4" containerName="extract"
Jul 07 18:02:16.013640 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.013620    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:16.016599 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.016575    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-f8xcc\""
Jul 07 18:02:16.026868 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.026842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"]
Jul 07 18:02:16.080832 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.080797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2g2d\" (UniqueName: \"kubernetes.io/projected/cb18c0ea-6a05-4439-b720-aeb19ba0f3bb-kube-api-access-n2g2d\") pod \"authorino-operator-75cc8d5cb7-tgcjv\" (UID: \"cb18c0ea-6a05-4439-b720-aeb19ba0f3bb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:16.182191 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.182150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2g2d\" (UniqueName: \"kubernetes.io/projected/cb18c0ea-6a05-4439-b720-aeb19ba0f3bb-kube-api-access-n2g2d\") pod \"authorino-operator-75cc8d5cb7-tgcjv\" (UID: \"cb18c0ea-6a05-4439-b720-aeb19ba0f3bb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:16.193529 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.193490    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2g2d\" (UniqueName: \"kubernetes.io/projected/cb18c0ea-6a05-4439-b720-aeb19ba0f3bb-kube-api-access-n2g2d\") pod \"authorino-operator-75cc8d5cb7-tgcjv\" (UID: \"cb18c0ea-6a05-4439-b720-aeb19ba0f3bb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:16.323872 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.323783    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:16.470812 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.470773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"]
Jul 07 18:02:16.474010 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:16.473982    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb18c0ea_6a05_4439_b720_aeb19ba0f3bb.slice/crio-e902a4f5993af1679996c31f81efe6e35ffcab1448fe714ae788874940c73e46 WatchSource:0}: Error finding container e902a4f5993af1679996c31f81efe6e35ffcab1448fe714ae788874940c73e46: Status 404 returned error can't find the container with id e902a4f5993af1679996c31f81efe6e35ffcab1448fe714ae788874940c73e46
Jul 07 18:02:16.826535 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:16.826499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv" event={"ID":"cb18c0ea-6a05-4439-b720-aeb19ba0f3bb","Type":"ContainerStarted","Data":"e902a4f5993af1679996c31f81efe6e35ffcab1448fe714ae788874940c73e46"}
Jul 07 18:02:17.283932 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.283873    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f758b7bf7-m5k8w"]
Jul 07 18:02:17.287185 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.287168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.381557 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.381518    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f758b7bf7-m5k8w"]
Jul 07 18:02:17.396017 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.395975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-trusted-ca-bundle\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.396017 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.396031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-oauth-config\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.396266 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.396058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-config\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.396266 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.396114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-serving-cert\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.396266 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.396141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bwfp\" (UniqueName: \"kubernetes.io/projected/01374afa-7bbb-44ff-81ba-eb1f21af8967-kube-api-access-9bwfp\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.396266 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.396191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-oauth-serving-cert\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.396266 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.396239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-service-ca\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-service-ca\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496997 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-trusted-ca-bundle\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496997 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-oauth-config\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496997 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-config\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496997 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-serving-cert\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496997 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bwfp\" (UniqueName: \"kubernetes.io/projected/01374afa-7bbb-44ff-81ba-eb1f21af8967-kube-api-access-9bwfp\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.496997 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.496949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-oauth-serving-cert\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.497620 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.497596    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-service-ca\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.497711 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.497600    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-oauth-serving-cert\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.497711 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.497651    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-config\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.498302 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.498254    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01374afa-7bbb-44ff-81ba-eb1f21af8967-trusted-ca-bundle\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.500305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.500256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-serving-cert\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.500305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.500262    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01374afa-7bbb-44ff-81ba-eb1f21af8967-console-oauth-config\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.512512 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.512479    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bwfp\" (UniqueName: \"kubernetes.io/projected/01374afa-7bbb-44ff-81ba-eb1f21af8967-kube-api-access-9bwfp\") pod \"console-6f758b7bf7-m5k8w\" (UID: \"01374afa-7bbb-44ff-81ba-eb1f21af8967\") " pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.597210 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.597162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:17.766074 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.766000    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f758b7bf7-m5k8w"]
Jul 07 18:02:17.768818 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:17.768773    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01374afa_7bbb_44ff_81ba_eb1f21af8967.slice/crio-6c021c51d3ebf041065bf43675e431eb693ce1edc7b4e5f388db6ed57ad5c5ff WatchSource:0}: Error finding container 6c021c51d3ebf041065bf43675e431eb693ce1edc7b4e5f388db6ed57ad5c5ff: Status 404 returned error can't find the container with id 6c021c51d3ebf041065bf43675e431eb693ce1edc7b4e5f388db6ed57ad5c5ff
Jul 07 18:02:17.834788 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:17.834753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f758b7bf7-m5k8w" event={"ID":"01374afa-7bbb-44ff-81ba-eb1f21af8967","Type":"ContainerStarted","Data":"6c021c51d3ebf041065bf43675e431eb693ce1edc7b4e5f388db6ed57ad5c5ff"}
Jul 07 18:02:18.845655 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.845616    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"]
Jul 07 18:02:18.851369 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.851265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:18.851724 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.851695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f758b7bf7-m5k8w" event={"ID":"01374afa-7bbb-44ff-81ba-eb1f21af8967","Type":"ContainerStarted","Data":"e5c6dfa8c47995d2fe4417abef8618ef4a292e0b24da625c42158b9595519716"}
Jul 07 18:02:18.852840 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.852815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"]
Jul 07 18:02:18.854306 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.854283    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 18:02:18.854452 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.854285    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cz2qq\""
Jul 07 18:02:18.915546 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:18.915488    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f758b7bf7-m5k8w" podStartSLOduration=1.9154644969999999 podStartE2EDuration="1.915464497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:18.913996015 +0000 UTC m=+500.804356739" watchObservedRunningTime="2026-07-07 18:02:18.915464497 +0000 UTC m=+500.805825221"
Jul 07 18:02:19.010867 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.010829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wf7\" (UniqueName: \"kubernetes.io/projected/17d61ef7-9e59-4c13-813e-f3fe2a5b685b-kube-api-access-72wf7\") pod \"dns-operator-controller-manager-864dd78c4c-mpg7w\" (UID: \"17d61ef7-9e59-4c13-813e-f3fe2a5b685b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:19.111896 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.111797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wf7\" (UniqueName: \"kubernetes.io/projected/17d61ef7-9e59-4c13-813e-f3fe2a5b685b-kube-api-access-72wf7\") pod \"dns-operator-controller-manager-864dd78c4c-mpg7w\" (UID: \"17d61ef7-9e59-4c13-813e-f3fe2a5b685b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:19.126398 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.126371    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wf7\" (UniqueName: \"kubernetes.io/projected/17d61ef7-9e59-4c13-813e-f3fe2a5b685b-kube-api-access-72wf7\") pod \"dns-operator-controller-manager-864dd78c4c-mpg7w\" (UID: \"17d61ef7-9e59-4c13-813e-f3fe2a5b685b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:19.166032 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.165992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:19.340135 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.339815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"]
Jul 07 18:02:19.858917 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.858867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w" event={"ID":"17d61ef7-9e59-4c13-813e-f3fe2a5b685b","Type":"ContainerStarted","Data":"0b0f77807e6293e9cfab4d112cd5ecdd4b4109df61ed487ddc8c4d2b393d8a5f"}
Jul 07 18:02:19.860560 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.860524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv" event={"ID":"cb18c0ea-6a05-4439-b720-aeb19ba0f3bb","Type":"ContainerStarted","Data":"4a1a901d012d6981ae48f0a7f3e2d40d3cba48d090072a7fe08fc996018bf240"}
Jul 07 18:02:19.904215 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:19.904161    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv" podStartSLOduration=2.396795697 podStartE2EDuration="4.904142082s" totalImagesPullingTime="2.507346385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:19.902396817 +0000 UTC m=+501.792757541" watchObservedRunningTime="2026-07-07 18:02:19.904142082 +0000 UTC m=+501.794502805"
Jul 07 18:02:20.866445 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:20.866412    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:21.872698 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.872662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w" event={"ID":"17d61ef7-9e59-4c13-813e-f3fe2a5b685b","Type":"ContainerStarted","Data":"feab29102d1e491ec0f00c2dd34c899a6677fc6b82099e640584e09cde1b9a7e"}
Jul 07 18:02:21.873148 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.872797    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:21.943812 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.943712    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"]
Jul 07 18:02:21.947631 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.947607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:21.952785 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.952761    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-n97qp\""
Jul 07 18:02:21.955533 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.955473    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w" podStartSLOduration=1.663377654 podStartE2EDuration="3.955450918s" totalImagesPullingTime="2.292073264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:21.952774507 +0000 UTC m=+503.843135246" watchObservedRunningTime="2026-07-07 18:02:21.955450918 +0000 UTC m=+503.845811645"
Jul 07 18:02:21.967393 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:21.967359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"]
Jul 07 18:02:22.037491 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:22.037451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cncnm\" (UniqueName: \"kubernetes.io/projected/32e9a27c-6274-4955-90d4-dc1e10c5552e-kube-api-access-cncnm\") pod \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" (UID: \"32e9a27c-6274-4955-90d4-dc1e10c5552e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:22.138479 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:22.138445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cncnm\" (UniqueName: \"kubernetes.io/projected/32e9a27c-6274-4955-90d4-dc1e10c5552e-kube-api-access-cncnm\") pod \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" (UID: \"32e9a27c-6274-4955-90d4-dc1e10c5552e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:22.168710 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:22.168677    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cncnm\" (UniqueName: \"kubernetes.io/projected/32e9a27c-6274-4955-90d4-dc1e10c5552e-kube-api-access-cncnm\") pod \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" (UID: \"32e9a27c-6274-4955-90d4-dc1e10c5552e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:22.259794 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:22.259690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:22.413942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:22.413738    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"]
Jul 07 18:02:22.416604 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:22.416576    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e9a27c_6274_4955_90d4_dc1e10c5552e.slice/crio-4d3ac684802c6fa366065b8b5456781b65768047816bb09f8d9cf863b2faa81b WatchSource:0}: Error finding container 4d3ac684802c6fa366065b8b5456781b65768047816bb09f8d9cf863b2faa81b: Status 404 returned error can't find the container with id 4d3ac684802c6fa366065b8b5456781b65768047816bb09f8d9cf863b2faa81b
Jul 07 18:02:22.879488 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:22.879456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g" event={"ID":"32e9a27c-6274-4955-90d4-dc1e10c5552e","Type":"ContainerStarted","Data":"4d3ac684802c6fa366065b8b5456781b65768047816bb09f8d9cf863b2faa81b"}
Jul 07 18:02:24.892147 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:24.892111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g" event={"ID":"32e9a27c-6274-4955-90d4-dc1e10c5552e","Type":"ContainerStarted","Data":"8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2"}
Jul 07 18:02:24.892537 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:24.892247    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:24.918486 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:24.918431    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g" podStartSLOduration=1.650679238 podStartE2EDuration="3.918414096s" totalImagesPullingTime="2.267734858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:24.918023164 +0000 UTC m=+506.808383889" watchObservedRunningTime="2026-07-07 18:02:24.918414096 +0000 UTC m=+506.808774819"
Jul 07 18:02:26.216139 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.216105    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"]
Jul 07 18:02:26.219712 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.219691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.223276 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.223250    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xjfgd\""
Jul 07 18:02:26.237490 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.237457    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"]
Jul 07 18:02:26.379719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.379680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.379719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.379721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2jdd\" (UniqueName: \"kubernetes.io/projected/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-kube-api-access-d2jdd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.480661 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.480568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.480661 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.480610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2jdd\" (UniqueName: \"kubernetes.io/projected/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-kube-api-access-d2jdd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.481034 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.481014    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.491532 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.491506    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2jdd\" (UniqueName: \"kubernetes.io/projected/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-kube-api-access-d2jdd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.530533 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.530489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:26.680031 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.680004    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"]
Jul 07 18:02:26.682568 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:26.682538    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7befcf5_38f6_411e_942a_ce2d8b7ce5c9.slice/crio-ca653b91060368e988ef19067c76179491db965ba92ea45d6ae2c7d3761c842b WatchSource:0}: Error finding container ca653b91060368e988ef19067c76179491db965ba92ea45d6ae2c7d3761c842b: Status 404 returned error can't find the container with id ca653b91060368e988ef19067c76179491db965ba92ea45d6ae2c7d3761c842b
Jul 07 18:02:26.904149 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:26.904115    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56" event={"ID":"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9","Type":"ContainerStarted","Data":"ca653b91060368e988ef19067c76179491db965ba92ea45d6ae2c7d3761c842b"}
Jul 07 18:02:27.597895 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:27.597856    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:27.598344 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:27.597955    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:27.603339 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:27.603315    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:27.919462 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:27.919377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f758b7bf7-m5k8w"
Jul 07 18:02:28.031152 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:28.031116    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-799f589db-t7p6j"]
Jul 07 18:02:31.875709 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:31.875673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-tgcjv"
Jul 07 18:02:32.882568 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:32.882536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mpg7w"
Jul 07 18:02:33.952857 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:33.952821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56" event={"ID":"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9","Type":"ContainerStarted","Data":"00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb"}
Jul 07 18:02:33.953293 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:33.952879    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:33.999127 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:33.999055    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56" podStartSLOduration=1.662105968 podStartE2EDuration="7.999036822s" totalImagesPullingTime="6.336930854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:33.996480088 +0000 UTC m=+515.886840812" watchObservedRunningTime="2026-07-07 18:02:33.999036822 +0000 UTC m=+515.889397546"
Jul 07 18:02:35.899214 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:35.899180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:44.960687 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:44.960654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:47.112265 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.112232    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"]
Jul 07 18:02:47.117266 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.117245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.144742 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.144707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"]
Jul 07 18:02:47.177749 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.177709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/100a8212-da44-4ee6-b380-fdd1357a56ea-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.177973 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.177778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbs49\" (UniqueName: \"kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.278407 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.278362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbs49\" (UniqueName: \"kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.278670 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.278448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/100a8212-da44-4ee6-b380-fdd1357a56ea-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.278849 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.278830    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/100a8212-da44-4ee6-b380-fdd1357a56ea-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.289638 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.289602    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"]
Jul 07 18:02:47.289898 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.289879    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tbs49], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7" podUID="100a8212-da44-4ee6-b380-fdd1357a56ea"
Jul 07 18:02:47.298175 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.298139    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"]
Jul 07 18:02:47.308701 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.308672    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tbs49 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7: failed to fetch token: pods "kuadrant-operator-controller-manager-55467d9f7-k4kk7" not found
Jul 07 18:02:47.308884 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.308746    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49 podName:100a8212-da44-4ee6-b380-fdd1357a56ea nodeName:}" failed. No retries permitted until 2026-07-07 18:02:47.80872496 +0000 UTC m=+529.699085665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tbs49" (UniqueName: "kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49") pod "kuadrant-operator-controller-manager-55467d9f7-k4kk7" (UID: "100a8212-da44-4ee6-b380-fdd1357a56ea") : failed to fetch token: pods "kuadrant-operator-controller-manager-55467d9f7-k4kk7" not found
Jul 07 18:02:47.329590 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.329549    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"]
Jul 07 18:02:47.329782 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.329745    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" containerName="manager" containerID="cri-o://00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb" gracePeriod=2
Jul 07 18:02:47.333166 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.333132    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"]
Jul 07 18:02:47.337080 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.337035    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:47.355189 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.355152    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"]
Jul 07 18:02:47.355449 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.355417    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" containerName="manager" containerID="cri-o://8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2" gracePeriod=2
Jul 07 18:02:47.383119 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.383018    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:47.383303 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.383247    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"]
Jul 07 18:02:47.384105 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.384070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" containerName="manager"
Jul 07 18:02:47.384105 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.384093    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" containerName="manager"
Jul 07 18:02:47.384297 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.384262    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" containerName="manager"
Jul 07 18:02:47.388319 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.388293    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"]
Jul 07 18:02:47.388440 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.388400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.396753 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.396709    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="0a7fed95-d81c-4962-90b5-c7c23f5b9504" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.398984 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.398957    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:47.422211 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.422163    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:47.481250 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.481214    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"]
Jul 07 18:02:47.486310 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.486278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b794z\" (UniqueName: \"kubernetes.io/projected/0a7fed95-d81c-4962-90b5-c7c23f5b9504-kube-api-access-b794z\") pod \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.486544 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.486511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a7fed95-d81c-4962-90b5-c7c23f5b9504-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.508930 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.508619    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"]
Jul 07 18:02:47.509554 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.509191    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" containerName="manager"
Jul 07 18:02:47.509554 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.509210    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" containerName="manager"
Jul 07 18:02:47.509554 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.509280    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" containerName="manager"
Jul 07 18:02:47.512556 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.512458    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"]
Jul 07 18:02:47.512656 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.512608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:02:47.515497 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.515457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.517303 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.517242    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:47.587340 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.587303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/038fe5ed-93a8-4511-927d-efada8106751-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pfrkw\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.587488 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.587363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5v96\" (UniqueName: \"kubernetes.io/projected/038fe5ed-93a8-4511-927d-efada8106751-kube-api-access-j5v96\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pfrkw\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.587488 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.587412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a7fed95-d81c-4962-90b5-c7c23f5b9504-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.587488 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.587461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shmvr\" (UniqueName: \"kubernetes.io/projected/f8f6024a-2301-4e0f-91df-9ee50965107f-kube-api-access-shmvr\") pod \"limitador-operator-controller-manager-859d7b78bd-hmwb2\" (UID: \"f8f6024a-2301-4e0f-91df-9ee50965107f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:02:47.587488 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.587487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b794z\" (UniqueName: \"kubernetes.io/projected/0a7fed95-d81c-4962-90b5-c7c23f5b9504-kube-api-access-b794z\") pod \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.588136 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.588103    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a7fed95-d81c-4962-90b5-c7c23f5b9504-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.588972 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.588952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"]
Jul 07 18:02:47.591348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.591321    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"]
Jul 07 18:02:47.591659 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.591632    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-b794z], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5" podUID="0a7fed95-d81c-4962-90b5-c7c23f5b9504"
Jul 07 18:02:47.618478 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.618448    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:47.643286 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.643260    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:47.687874 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.687831    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2jdd\" (UniqueName: \"kubernetes.io/projected/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-kube-api-access-d2jdd\") pod \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") "
Jul 07 18:02:47.688104 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.687893    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-extensions-socket-volume\") pod \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\" (UID: \"f7befcf5-38f6-411e-942a-ce2d8b7ce5c9\") "
Jul 07 18:02:47.688104 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.688022    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cncnm\" (UniqueName: \"kubernetes.io/projected/32e9a27c-6274-4955-90d4-dc1e10c5552e-kube-api-access-cncnm\") pod \"32e9a27c-6274-4955-90d4-dc1e10c5552e\" (UID: \"32e9a27c-6274-4955-90d4-dc1e10c5552e\") "
Jul 07 18:02:47.688213 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.688176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shmvr\" (UniqueName: \"kubernetes.io/projected/f8f6024a-2301-4e0f-91df-9ee50965107f-kube-api-access-shmvr\") pod \"limitador-operator-controller-manager-859d7b78bd-hmwb2\" (UID: \"f8f6024a-2301-4e0f-91df-9ee50965107f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:02:47.688299 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.688283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/038fe5ed-93a8-4511-927d-efada8106751-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pfrkw\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.688358 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.688319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5v96\" (UniqueName: \"kubernetes.io/projected/038fe5ed-93a8-4511-927d-efada8106751-kube-api-access-j5v96\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pfrkw\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.688429 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.688403    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" (UID: "f7befcf5-38f6-411e-942a-ce2d8b7ce5c9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:47.688684 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.688665    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/038fe5ed-93a8-4511-927d-efada8106751-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pfrkw\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.690299 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.690272    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-kube-api-access-d2jdd" (OuterVolumeSpecName: "kube-api-access-d2jdd") pod "f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" (UID: "f7befcf5-38f6-411e-942a-ce2d8b7ce5c9"). InnerVolumeSpecName "kube-api-access-d2jdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:47.690375 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.690279    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e9a27c-6274-4955-90d4-dc1e10c5552e-kube-api-access-cncnm" (OuterVolumeSpecName: "kube-api-access-cncnm") pod "32e9a27c-6274-4955-90d4-dc1e10c5552e" (UID: "32e9a27c-6274-4955-90d4-dc1e10c5552e"). InnerVolumeSpecName "kube-api-access-cncnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:47.789068 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.789032    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cncnm\" (UniqueName: \"kubernetes.io/projected/32e9a27c-6274-4955-90d4-dc1e10c5552e-kube-api-access-cncnm\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:47.789068 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.789058    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2jdd\" (UniqueName: \"kubernetes.io/projected/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-kube-api-access-d2jdd\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:47.789068 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.789067    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9-extensions-socket-volume\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:47.824079 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.824041    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shmvr\" (UniqueName: \"kubernetes.io/projected/f8f6024a-2301-4e0f-91df-9ee50965107f-kube-api-access-shmvr\") pod \"limitador-operator-controller-manager-859d7b78bd-hmwb2\" (UID: \"f8f6024a-2301-4e0f-91df-9ee50965107f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:02:47.824707 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.824690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:02:47.842899 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.842863    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b794z\" (UniqueName: \"kubernetes.io/projected/0a7fed95-d81c-4962-90b5-c7c23f5b9504-kube-api-access-b794z\") pod \"kuadrant-operator-controller-manager-55467d9f7-b2cz5\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:47.863316 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.863265    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:47.882092 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.882014    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5v96\" (UniqueName: \"kubernetes.io/projected/038fe5ed-93a8-4511-927d-efada8106751-kube-api-access-j5v96\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pfrkw\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:47.888107 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.888031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"]
Jul 07 18:02:47.890470 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:47.890267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbs49\" (UniqueName: \"kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:47.958929 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.958879    2580 projected.go:196] Error preparing data for projected volume kube-api-access-tbs49 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-k4kk7" not found
Jul 07 18:02:47.959082 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:47.958981    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49 podName:100a8212-da44-4ee6-b380-fdd1357a56ea nodeName:}" failed. No retries permitted until 2026-07-07 18:02:48.95896245 +0000 UTC m=+530.849323153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbs49" (UniqueName: "kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49") pod "kuadrant-operator-controller-manager-55467d9f7-k4kk7" (UID: "100a8212-da44-4ee6-b380-fdd1357a56ea") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-k4kk7" not found
Jul 07 18:02:48.035873 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.035834    2580 generic.go:356] "Generic (PLEG): container finished" podID="32e9a27c-6274-4955-90d4-dc1e10c5552e" containerID="8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2" exitCode=0
Jul 07 18:02:48.036080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.035887    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:48.036080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.035958    2580 scope.go:117] "RemoveContainer" containerID="8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2"
Jul 07 18:02:48.037540 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.037494    2580 generic.go:356] "Generic (PLEG): container finished" podID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" containerID="00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb" exitCode=0
Jul 07 18:02:48.037660 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.037545    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:48.037726 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.037686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:48.037831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.037816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:48.048160 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.048139    2580 scope.go:117] "RemoveContainer" containerID="8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2"
Jul 07 18:02:48.048463 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.048445    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2\": container with ID starting with 8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2 not found: ID does not exist" containerID="8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2"
Jul 07 18:02:48.048521 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.048474    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2"} err="failed to get container status \"8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2\": rpc error: code = NotFound desc = could not find container \"8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2\": container with ID starting with 8ab62a0083e7676fcdf9a052a1de358893de689571c846c3e554a23db5bb97e2 not found: ID does not exist"
Jul 07 18:02:48.048521 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.048495    2580 scope.go:117] "RemoveContainer" containerID="00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb"
Jul 07 18:02:48.049588 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.049571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:48.057362 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.057339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:48.058120 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.058101    2580 scope.go:117] "RemoveContainer" containerID="00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb"
Jul 07 18:02:48.058384 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.058365    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb\": container with ID starting with 00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb not found: ID does not exist" containerID="00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb"
Jul 07 18:02:48.058439 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.058393    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb"} err="failed to get container status \"00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb\": rpc error: code = NotFound desc = could not find container \"00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb\": container with ID starting with 00e7c3b086a3c09e9225146be3ff52bf0f8378f5c7056963ab472ad623cd4dfb not found: ID does not exist"
Jul 07 18:02:48.070507 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.070465    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:48.076383 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.076351    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:48.091596 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.091562    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/100a8212-da44-4ee6-b380-fdd1357a56ea-extensions-socket-volume\") pod \"100a8212-da44-4ee6-b380-fdd1357a56ea\" (UID: \"100a8212-da44-4ee6-b380-fdd1357a56ea\") "
Jul 07 18:02:48.091823 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.091807    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbs49\" (UniqueName: \"kubernetes.io/projected/100a8212-da44-4ee6-b380-fdd1357a56ea-kube-api-access-tbs49\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:48.091898 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.091858    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/100a8212-da44-4ee6-b380-fdd1357a56ea-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "100a8212-da44-4ee6-b380-fdd1357a56ea" (UID: "100a8212-da44-4ee6-b380-fdd1357a56ea"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:48.100810 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:48.100783    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f6024a_2301_4e0f_91df_9ee50965107f.slice/crio-aff1624debecfa8e14987ab5f02c43a51220e16b1a0b6f8382632e67ee8e1e6f WatchSource:0}: Error finding container aff1624debecfa8e14987ab5f02c43a51220e16b1a0b6f8382632e67ee8e1e6f: Status 404 returned error can't find the container with id aff1624debecfa8e14987ab5f02c43a51220e16b1a0b6f8382632e67ee8e1e6f
Jul 07 18:02:48.130892 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.130854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:48.140802 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.140736    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:48.143586 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.143557    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:48.146432 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.146400    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="100a8212-da44-4ee6-b380-fdd1357a56ea" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:48.146576 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.146556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"]
Jul 07 18:02:48.193485 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.192375    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a7fed95-d81c-4962-90b5-c7c23f5b9504-extensions-socket-volume\") pod \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") "
Jul 07 18:02:48.193485 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.192510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b794z\" (UniqueName: \"kubernetes.io/projected/0a7fed95-d81c-4962-90b5-c7c23f5b9504-kube-api-access-b794z\") pod \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\" (UID: \"0a7fed95-d81c-4962-90b5-c7c23f5b9504\") "
Jul 07 18:02:48.193485 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.192748    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/100a8212-da44-4ee6-b380-fdd1357a56ea-extensions-socket-volume\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:48.194233 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.194198    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a7fed95-d81c-4962-90b5-c7c23f5b9504-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a7fed95-d81c-4962-90b5-c7c23f5b9504" (UID: "0a7fed95-d81c-4962-90b5-c7c23f5b9504"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:02:48.196087 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.196064    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a7fed95-d81c-4962-90b5-c7c23f5b9504-kube-api-access-b794z" (OuterVolumeSpecName: "kube-api-access-b794z") pod "0a7fed95-d81c-4962-90b5-c7c23f5b9504" (UID: "0a7fed95-d81c-4962-90b5-c7c23f5b9504"). InnerVolumeSpecName "kube-api-access-b794z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:48.294134 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.294094    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a7fed95-d81c-4962-90b5-c7c23f5b9504-extensions-socket-volume\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:48.294134 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.294130    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b794z\" (UniqueName: \"kubernetes.io/projected/0a7fed95-d81c-4962-90b5-c7c23f5b9504-kube-api-access-b794z\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:48.341950 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:02:48.341895    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038fe5ed_93a8_4511_927d_efada8106751.slice/crio-2d29c7af9fde748568b38a2f072b6c779a8a51ba108506f24f3d4de88abc45e9 WatchSource:0}: Error finding container 2d29c7af9fde748568b38a2f072b6c779a8a51ba108506f24f3d4de88abc45e9: Status 404 returned error can't find the container with id 2d29c7af9fde748568b38a2f072b6c779a8a51ba108506f24f3d4de88abc45e9
Jul 07 18:02:48.352062 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.352035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"]
Jul 07 18:02:48.595215 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.595187    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="100a8212-da44-4ee6-b380-fdd1357a56ea" path="/var/lib/kubelet/pods/100a8212-da44-4ee6-b380-fdd1357a56ea/volumes"
Jul 07 18:02:48.595428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.595416    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" path="/var/lib/kubelet/pods/32e9a27c-6274-4955-90d4-dc1e10c5552e/volumes"
Jul 07 18:02:48.595724 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:48.595712    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" path="/var/lib/kubelet/pods/f7befcf5-38f6-411e-942a-ce2d8b7ce5c9/volumes"
Jul 07 18:02:48.597247 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.597215    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7kr4g\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="32e9a27c-6274-4955-90d4-dc1e10c5552e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7kr4g"
Jul 07 18:02:48.599484 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.599461    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="100a8212-da44-4ee6-b380-fdd1357a56ea" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:48.602063 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:48.602036    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c8n56\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="f7befcf5-38f6-411e-942a-ce2d8b7ce5c9" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c8n56"
Jul 07 18:02:49.044503 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.044468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2" event={"ID":"f8f6024a-2301-4e0f-91df-9ee50965107f","Type":"ContainerStarted","Data":"6eceec015fef893da8037254791d79dff15938de00a960f2e0f874030ddc438b"}
Jul 07 18:02:49.044716 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.044525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2" event={"ID":"f8f6024a-2301-4e0f-91df-9ee50965107f","Type":"ContainerStarted","Data":"aff1624debecfa8e14987ab5f02c43a51220e16b1a0b6f8382632e67ee8e1e6f"}
Jul 07 18:02:49.044716 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.044584    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:02:49.047009 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.046980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw" event={"ID":"038fe5ed-93a8-4511-927d-efada8106751","Type":"ContainerStarted","Data":"139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239"}
Jul 07 18:02:49.047164 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.047026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw" event={"ID":"038fe5ed-93a8-4511-927d-efada8106751","Type":"ContainerStarted","Data":"2d29c7af9fde748568b38a2f072b6c779a8a51ba108506f24f3d4de88abc45e9"}
Jul 07 18:02:49.047164 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.047067    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:02:49.048014 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.047996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"
Jul 07 18:02:49.048120 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.047996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:49.092306 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.092247    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2" podStartSLOduration=2.092229453 podStartE2EDuration="2.092229453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:49.088727787 +0000 UTC m=+530.979088513" watchObservedRunningTime="2026-07-07 18:02:49.092229453 +0000 UTC m=+530.982590177"
Jul 07 18:02:49.102326 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:49.102292    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-k4kk7\" is forbidden: User \"system:node:ip-10-0-134-243.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-243.ec2.internal' and this object" podUID="100a8212-da44-4ee6-b380-fdd1357a56ea" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4kk7"
Jul 07 18:02:49.167761 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.167721    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"]
Jul 07 18:02:49.180745 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.180711    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b2cz5"]
Jul 07 18:02:49.205925 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:49.205851    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw" podStartSLOduration=2.205836348 podStartE2EDuration="2.205836348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:02:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:02:49.204063511 +0000 UTC m=+531.094424236" watchObservedRunningTime="2026-07-07 18:02:49.205836348 +0000 UTC m=+531.096197072"
Jul 07 18:02:50.595462 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:50.595422    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7fed95-d81c-4962-90b5-c7c23f5b9504" path="/var/lib/kubelet/pods/0a7fed95-d81c-4962-90b5-c7c23f5b9504/volumes"
Jul 07 18:02:53.052305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.052256    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-799f589db-t7p6j" podUID="246813f9-a50d-4ded-bb9a-89c00bc7ea54" containerName="console" containerID="cri-o://3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2" gracePeriod=15
Jul 07 18:02:53.296391 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.296366    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 18:02:53.441720 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441659    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-trusted-ca-bundle\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.441720 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-oauth-config\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.442027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-serving-cert\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.442027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-oauth-serving-cert\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.442027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441826    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-service-ca\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.442027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-config\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.442027 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.441953    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67lk6\" (UniqueName: \"kubernetes.io/projected/246813f9-a50d-4ded-bb9a-89c00bc7ea54-kube-api-access-67lk6\") pod \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\" (UID: \"246813f9-a50d-4ded-bb9a-89c00bc7ea54\") "
Jul 07 18:02:53.442343 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.442209    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 18:02:53.442437 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.442410    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-service-ca" (OuterVolumeSpecName: "service-ca") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 18:02:53.442437 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.442425    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 18:02:53.442555 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.442445    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-config" (OuterVolumeSpecName: "console-config") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 18:02:53.444168 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.444141    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:02:53.444269 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.444211    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:02:53.444335 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.444272    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/246813f9-a50d-4ded-bb9a-89c00bc7ea54-kube-api-access-67lk6" (OuterVolumeSpecName: "kube-api-access-67lk6") pod "246813f9-a50d-4ded-bb9a-89c00bc7ea54" (UID: "246813f9-a50d-4ded-bb9a-89c00bc7ea54"). InnerVolumeSpecName "kube-api-access-67lk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:02:53.543563 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543521    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67lk6\" (UniqueName: \"kubernetes.io/projected/246813f9-a50d-4ded-bb9a-89c00bc7ea54-kube-api-access-67lk6\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:53.543563 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543554    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-trusted-ca-bundle\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:53.543563 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543564    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-oauth-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:53.543563 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543573    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:53.543837 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543582    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-oauth-serving-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:53.543837 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543591    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-service-ca\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:53.543837 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:53.543599    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/246813f9-a50d-4ded-bb9a-89c00bc7ea54-console-config\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:02:54.078719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.078681    2580 generic.go:356] "Generic (PLEG): container finished" podID="246813f9-a50d-4ded-bb9a-89c00bc7ea54" containerID="3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2" exitCode=0
Jul 07 18:02:54.079154 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.078731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799f589db-t7p6j" event={"ID":"246813f9-a50d-4ded-bb9a-89c00bc7ea54","Type":"ContainerDied","Data":"3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2"}
Jul 07 18:02:54.079154 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.078738    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-799f589db-t7p6j"
Jul 07 18:02:54.079154 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.078802    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-799f589db-t7p6j" event={"ID":"246813f9-a50d-4ded-bb9a-89c00bc7ea54","Type":"ContainerDied","Data":"a4350d67d4975f1b4c057fab15e24b9a178d813b036ecd88ab2b0e4b49ab2fd7"}
Jul 07 18:02:54.079154 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.078824    2580 scope.go:117] "RemoveContainer" containerID="3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2"
Jul 07 18:02:54.088282 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.088258    2580 scope.go:117] "RemoveContainer" containerID="3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2"
Jul 07 18:02:54.088715 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:02:54.088692    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2\": container with ID starting with 3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2 not found: ID does not exist" containerID="3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2"
Jul 07 18:02:54.088805 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.088723    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2"} err="failed to get container status \"3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2\": rpc error: code = NotFound desc = could not find container \"3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2\": container with ID starting with 3c7119f1d24077c6f0d2e1d6cc1c5bafa7bf6fd42b0ac7cee99799446f7672f2 not found: ID does not exist"
Jul 07 18:02:54.109985 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.109942    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-799f589db-t7p6j"]
Jul 07 18:02:54.124196 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.124160    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-799f589db-t7p6j"]
Jul 07 18:02:54.595827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:02:54.595787    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="246813f9-a50d-4ded-bb9a-89c00bc7ea54" path="/var/lib/kubelet/pods/246813f9-a50d-4ded-bb9a-89c00bc7ea54/volumes"
Jul 07 18:03:00.055642 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:00.055610    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:03:00.056029 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:00.055684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hmwb2"
Jul 07 18:03:19.565853 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.565810    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"]
Jul 07 18:03:19.566871 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.566820    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw" podUID="038fe5ed-93a8-4511-927d-efada8106751" containerName="manager" containerID="cri-o://139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239" gracePeriod=10
Jul 07 18:03:19.829748 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.829682    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:03:19.874431 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.874394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5v96\" (UniqueName: \"kubernetes.io/projected/038fe5ed-93a8-4511-927d-efada8106751-kube-api-access-j5v96\") pod \"038fe5ed-93a8-4511-927d-efada8106751\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") "
Jul 07 18:03:19.874615 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.874483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/038fe5ed-93a8-4511-927d-efada8106751-extensions-socket-volume\") pod \"038fe5ed-93a8-4511-927d-efada8106751\" (UID: \"038fe5ed-93a8-4511-927d-efada8106751\") "
Jul 07 18:03:19.874876 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.874843    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/038fe5ed-93a8-4511-927d-efada8106751-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "038fe5ed-93a8-4511-927d-efada8106751" (UID: "038fe5ed-93a8-4511-927d-efada8106751"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:03:19.876732 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.876709    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/038fe5ed-93a8-4511-927d-efada8106751-kube-api-access-j5v96" (OuterVolumeSpecName: "kube-api-access-j5v96") pod "038fe5ed-93a8-4511-927d-efada8106751" (UID: "038fe5ed-93a8-4511-927d-efada8106751"). InnerVolumeSpecName "kube-api-access-j5v96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:03:19.975717 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.975664    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/038fe5ed-93a8-4511-927d-efada8106751-extensions-socket-volume\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:03:19.975717 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:19.975701    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5v96\" (UniqueName: \"kubernetes.io/projected/038fe5ed-93a8-4511-927d-efada8106751-kube-api-access-j5v96\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:03:20.228442 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.228405    2580 generic.go:356] "Generic (PLEG): container finished" podID="038fe5ed-93a8-4511-927d-efada8106751" containerID="139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239" exitCode=0
Jul 07 18:03:20.228642 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.228451    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"
Jul 07 18:03:20.228642 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.228499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw" event={"ID":"038fe5ed-93a8-4511-927d-efada8106751","Type":"ContainerDied","Data":"139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239"}
Jul 07 18:03:20.228642 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.228563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw" event={"ID":"038fe5ed-93a8-4511-927d-efada8106751","Type":"ContainerDied","Data":"2d29c7af9fde748568b38a2f072b6c779a8a51ba108506f24f3d4de88abc45e9"}
Jul 07 18:03:20.228642 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.228588    2580 scope.go:117] "RemoveContainer" containerID="139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239"
Jul 07 18:03:20.237863 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.237846    2580 scope.go:117] "RemoveContainer" containerID="139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239"
Jul 07 18:03:20.238181 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:03:20.238161    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239\": container with ID starting with 139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239 not found: ID does not exist" containerID="139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239"
Jul 07 18:03:20.238259 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.238189    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239"} err="failed to get container status \"139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239\": rpc error: code = NotFound desc = could not find container \"139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239\": container with ID starting with 139bbdb8b6b41331ad92a8d94b5ece49ce2d9a7203873fb8fdfac3f2100f3239 not found: ID does not exist"
Jul 07 18:03:20.252964 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.252934    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"]
Jul 07 18:03:20.256762 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.256730    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pfrkw"]
Jul 07 18:03:20.595728 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:20.595698    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="038fe5ed-93a8-4511-927d-efada8106751" path="/var/lib/kubelet/pods/038fe5ed-93a8-4511-927d-efada8106751/volumes"
Jul 07 18:03:36.054201 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054163    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"]
Jul 07 18:03:36.054756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054519    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="038fe5ed-93a8-4511-927d-efada8106751" containerName="manager"
Jul 07 18:03:36.054756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054530    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="038fe5ed-93a8-4511-927d-efada8106751" containerName="manager"
Jul 07 18:03:36.054756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054544    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="246813f9-a50d-4ded-bb9a-89c00bc7ea54" containerName="console"
Jul 07 18:03:36.054756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054550    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="246813f9-a50d-4ded-bb9a-89c00bc7ea54" containerName="console"
Jul 07 18:03:36.054756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054604    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="246813f9-a50d-4ded-bb9a-89c00bc7ea54" containerName="console"
Jul 07 18:03:36.054756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.054613    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="038fe5ed-93a8-4511-927d-efada8106751" containerName="manager"
Jul 07 18:03:36.059183 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.059164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.062366 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.062335    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 18:03:36.062604 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.062590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vzlsd\""
Jul 07 18:03:36.073007 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.072981    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"]
Jul 07 18:03:36.111383 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422134db-9391-4170-a04a-613ea85b8ef0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111768 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422134db-9391-4170-a04a-613ea85b8ef0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111768 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422134db-9391-4170-a04a-613ea85b8ef0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.111768 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.111672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxwv\" (UniqueName: \"kubernetes.io/projected/422134db-9391-4170-a04a-613ea85b8ef0-kube-api-access-gcxwv\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212362 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422134db-9391-4170-a04a-613ea85b8ef0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212362 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422134db-9391-4170-a04a-613ea85b8ef0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212605 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxwv\" (UniqueName: \"kubernetes.io/projected/422134db-9391-4170-a04a-613ea85b8ef0-kube-api-access-gcxwv\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212605 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422134db-9391-4170-a04a-613ea85b8ef0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212605 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212605 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212605 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.212605 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.212586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.213796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.213136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.213796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.213397    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.213796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.213417    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.213796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.213628    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.213796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.213661    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.213796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.213771    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422134db-9391-4170-a04a-613ea85b8ef0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.217298 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.217267    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422134db-9391-4170-a04a-613ea85b8ef0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.222655 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.222624    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422134db-9391-4170-a04a-613ea85b8ef0-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.223776 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.223751    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422134db-9391-4170-a04a-613ea85b8ef0-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.223872 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.223837    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxwv\" (UniqueName: \"kubernetes.io/projected/422134db-9391-4170-a04a-613ea85b8ef0-kube-api-access-gcxwv\") pod \"maas-default-gateway-openshift-default-56654dcf7-49h8f\" (UID: \"422134db-9391-4170-a04a-613ea85b8ef0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.369972 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.369931    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:36.519454 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:36.519427    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"]
Jul 07 18:03:36.521336 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:03:36.521309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422134db_9391_4170_a04a_613ea85b8ef0.slice/crio-8ddaf57d8654fcc2cb7e90566c34b189203a412acb1ef4750858b4f94eed3f4d WatchSource:0}: Error finding container 8ddaf57d8654fcc2cb7e90566c34b189203a412acb1ef4750858b4f94eed3f4d: Status 404 returned error can't find the container with id 8ddaf57d8654fcc2cb7e90566c34b189203a412acb1ef4750858b4f94eed3f4d
Jul 07 18:03:37.316338 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:37.316304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" event={"ID":"422134db-9391-4170-a04a-613ea85b8ef0","Type":"ContainerStarted","Data":"8ddaf57d8654fcc2cb7e90566c34b189203a412acb1ef4750858b4f94eed3f4d"}
Jul 07 18:03:41.378594 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:41.378547    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:03:41.378836 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:41.378636    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:03:41.378836 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:41.378664    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:03:42.343669 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:42.343628    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" event={"ID":"422134db-9391-4170-a04a-613ea85b8ef0","Type":"ContainerStarted","Data":"a9a48f16c8194e5049978561f249f83aadcb9804ff5680d41c408d74ed4096db"}
Jul 07 18:03:42.367416 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:42.367367    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" podStartSLOduration=1.512436157 podStartE2EDuration="6.36734928s" totalImagesPullingTime="4.854913123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:03:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:03:42.365379864 +0000 UTC m=+584.255740591" watchObservedRunningTime="2026-07-07 18:03:42.36734928 +0000 UTC m=+584.257710003"
Jul 07 18:03:42.370662 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:42.370642    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:42.375292 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:42.375271    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:43.349040 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.349006    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:43.350085 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.350060    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:03:43.463730 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.463683    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:43.467533 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.467506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.470993 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.470957    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 18:03:43.471157 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.471045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xxlhd\""
Jul 07 18:03:43.479855 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.479830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:43.575928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.575884    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:43.583804 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.583774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g26dh\" (UniqueName: \"kubernetes.io/projected/4eab7e38-a04a-4690-b80a-ffe568a6e56e-kube-api-access-g26dh\") pod \"limitador-limitador-64c94fd64f-75kpr\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.583963 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.583817    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4eab7e38-a04a-4690-b80a-ffe568a6e56e-config-file\") pod \"limitador-limitador-64c94fd64f-75kpr\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.685168 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.685081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g26dh\" (UniqueName: \"kubernetes.io/projected/4eab7e38-a04a-4690-b80a-ffe568a6e56e-kube-api-access-g26dh\") pod \"limitador-limitador-64c94fd64f-75kpr\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.685168 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.685125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4eab7e38-a04a-4690-b80a-ffe568a6e56e-config-file\") pod \"limitador-limitador-64c94fd64f-75kpr\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.685719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.685702    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4eab7e38-a04a-4690-b80a-ffe568a6e56e-config-file\") pod \"limitador-limitador-64c94fd64f-75kpr\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.693938 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.693890    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g26dh\" (UniqueName: \"kubernetes.io/projected/4eab7e38-a04a-4690-b80a-ffe568a6e56e-kube-api-access-g26dh\") pod \"limitador-limitador-64c94fd64f-75kpr\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.778359 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.778329    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:43.928351 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:43.928281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:43.931137 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:03:43.931111    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eab7e38_a04a_4690_b80a_ffe568a6e56e.slice/crio-3121047b1ae5975bab3c84d9ba5c9e50142279ac2208353e6cfe4c3738d5a463 WatchSource:0}: Error finding container 3121047b1ae5975bab3c84d9ba5c9e50142279ac2208353e6cfe4c3738d5a463: Status 404 returned error can't find the container with id 3121047b1ae5975bab3c84d9ba5c9e50142279ac2208353e6cfe4c3738d5a463
Jul 07 18:03:44.275097 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.275003    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-22666"]
Jul 07 18:03:44.279665 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.279643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:44.282294 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.282269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bdkx7\""
Jul 07 18:03:44.286923 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.286882    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-22666"]
Jul 07 18:03:44.354571 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.354536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr" event={"ID":"4eab7e38-a04a-4690-b80a-ffe568a6e56e","Type":"ContainerStarted","Data":"3121047b1ae5975bab3c84d9ba5c9e50142279ac2208353e6cfe4c3738d5a463"}
Jul 07 18:03:44.391536 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.391503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7sbv\" (UniqueName: \"kubernetes.io/projected/f88307b0-95cd-4230-b378-8b00c0e157f9-kube-api-access-q7sbv\") pod \"authorino-58cd955675-22666\" (UID: \"f88307b0-95cd-4230-b378-8b00c0e157f9\") " pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:44.492022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.491990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7sbv\" (UniqueName: \"kubernetes.io/projected/f88307b0-95cd-4230-b378-8b00c0e157f9-kube-api-access-q7sbv\") pod \"authorino-58cd955675-22666\" (UID: \"f88307b0-95cd-4230-b378-8b00c0e157f9\") " pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:44.500689 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.500654    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7sbv\" (UniqueName: \"kubernetes.io/projected/f88307b0-95cd-4230-b378-8b00c0e157f9-kube-api-access-q7sbv\") pod \"authorino-58cd955675-22666\" (UID: \"f88307b0-95cd-4230-b378-8b00c0e157f9\") " pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:44.590589 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.590500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:44.809479 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:44.808265    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-22666"]
Jul 07 18:03:44.814532 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:03:44.814499    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf88307b0_95cd_4230_b378_8b00c0e157f9.slice/crio-40badac743aba9151e6733b594e2a86a400dc0f1933c1de50f1b6a8e8577176e WatchSource:0}: Error finding container 40badac743aba9151e6733b594e2a86a400dc0f1933c1de50f1b6a8e8577176e: Status 404 returned error can't find the container with id 40badac743aba9151e6733b594e2a86a400dc0f1933c1de50f1b6a8e8577176e
Jul 07 18:03:45.363395 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:45.363359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-22666" event={"ID":"f88307b0-95cd-4230-b378-8b00c0e157f9","Type":"ContainerStarted","Data":"40badac743aba9151e6733b594e2a86a400dc0f1933c1de50f1b6a8e8577176e"}
Jul 07 18:03:48.701238 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:48.701203    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-22666"]
Jul 07 18:03:49.398404 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:49.398363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-22666" event={"ID":"f88307b0-95cd-4230-b378-8b00c0e157f9","Type":"ContainerStarted","Data":"79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13"}
Jul 07 18:03:49.399694 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:49.399670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr" event={"ID":"4eab7e38-a04a-4690-b80a-ffe568a6e56e","Type":"ContainerStarted","Data":"fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1"}
Jul 07 18:03:49.399823 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:49.399802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:49.420783 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:49.420737    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-22666" podStartSLOduration=1.6828079759999999 podStartE2EDuration="5.42072103s" totalImagesPullingTime="3.737913054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:03:49.418256977 +0000 UTC m=+591.308617701" watchObservedRunningTime="2026-07-07 18:03:49.42072103 +0000 UTC m=+591.311081754"
Jul 07 18:03:49.438050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:49.437996    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr" podStartSLOduration=1.824129721 podStartE2EDuration="6.437977288s" totalImagesPullingTime="4.613847567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:03:49.435475464 +0000 UTC m=+591.325836189" watchObservedRunningTime="2026-07-07 18:03:49.437977288 +0000 UTC m=+591.328338011"
Jul 07 18:03:50.404684 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:50.404641    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-22666" podUID="f88307b0-95cd-4230-b378-8b00c0e157f9" containerName="authorino" containerID="cri-o://79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13" gracePeriod=30
Jul 07 18:03:50.651153 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:50.651129    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:50.763365 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:50.763275    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7sbv\" (UniqueName: \"kubernetes.io/projected/f88307b0-95cd-4230-b378-8b00c0e157f9-kube-api-access-q7sbv\") pod \"f88307b0-95cd-4230-b378-8b00c0e157f9\" (UID: \"f88307b0-95cd-4230-b378-8b00c0e157f9\") "
Jul 07 18:03:50.765528 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:50.765505    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88307b0-95cd-4230-b378-8b00c0e157f9-kube-api-access-q7sbv" (OuterVolumeSpecName: "kube-api-access-q7sbv") pod "f88307b0-95cd-4230-b378-8b00c0e157f9" (UID: "f88307b0-95cd-4230-b378-8b00c0e157f9"). InnerVolumeSpecName "kube-api-access-q7sbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:03:50.864810 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:50.864768    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7sbv\" (UniqueName: \"kubernetes.io/projected/f88307b0-95cd-4230-b378-8b00c0e157f9-kube-api-access-q7sbv\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:03:51.411122 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.411088    2580 generic.go:356] "Generic (PLEG): container finished" podID="f88307b0-95cd-4230-b378-8b00c0e157f9" containerID="79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13" exitCode=0
Jul 07 18:03:51.411597 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.411142    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-22666"
Jul 07 18:03:51.411597 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.411175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-22666" event={"ID":"f88307b0-95cd-4230-b378-8b00c0e157f9","Type":"ContainerDied","Data":"79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13"}
Jul 07 18:03:51.411597 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.411219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-22666" event={"ID":"f88307b0-95cd-4230-b378-8b00c0e157f9","Type":"ContainerDied","Data":"40badac743aba9151e6733b594e2a86a400dc0f1933c1de50f1b6a8e8577176e"}
Jul 07 18:03:51.411597 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.411237    2580 scope.go:117] "RemoveContainer" containerID="79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13"
Jul 07 18:03:51.420482 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.420461    2580 scope.go:117] "RemoveContainer" containerID="79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13"
Jul 07 18:03:51.420767 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:03:51.420746    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13\": container with ID starting with 79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13 not found: ID does not exist" containerID="79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13"
Jul 07 18:03:51.420872 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.420775    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13"} err="failed to get container status \"79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13\": rpc error: code = NotFound desc = could not find container \"79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13\": container with ID starting with 79edbf7d5403b00cee3495abc66e8bf142b7f23dc0010831b5955a144a68ca13 not found: ID does not exist"
Jul 07 18:03:51.433054 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.433016    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-22666"]
Jul 07 18:03:51.437360 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:51.437330    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-22666"]
Jul 07 18:03:52.595068 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:52.595037    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88307b0-95cd-4230-b378-8b00c0e157f9" path="/var/lib/kubelet/pods/f88307b0-95cd-4230-b378-8b00c0e157f9/volumes"
Jul 07 18:03:58.259352 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.259313    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:58.259875 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.259514    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr" podUID="4eab7e38-a04a-4690-b80a-ffe568a6e56e" containerName="limitador" containerID="cri-o://fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1" gracePeriod=30
Jul 07 18:03:58.260298 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.260255    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:58.811850 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.811790    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:58.939192 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.939149    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g26dh\" (UniqueName: \"kubernetes.io/projected/4eab7e38-a04a-4690-b80a-ffe568a6e56e-kube-api-access-g26dh\") pod \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") "
Jul 07 18:03:58.939192 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.939216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4eab7e38-a04a-4690-b80a-ffe568a6e56e-config-file\") pod \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\" (UID: \"4eab7e38-a04a-4690-b80a-ffe568a6e56e\") "
Jul 07 18:03:58.939584 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.939559    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eab7e38-a04a-4690-b80a-ffe568a6e56e-config-file" (OuterVolumeSpecName: "config-file") pod "4eab7e38-a04a-4690-b80a-ffe568a6e56e" (UID: "4eab7e38-a04a-4690-b80a-ffe568a6e56e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 18:03:58.941634 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:58.941599    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eab7e38-a04a-4690-b80a-ffe568a6e56e-kube-api-access-g26dh" (OuterVolumeSpecName: "kube-api-access-g26dh") pod "4eab7e38-a04a-4690-b80a-ffe568a6e56e" (UID: "4eab7e38-a04a-4690-b80a-ffe568a6e56e"). InnerVolumeSpecName "kube-api-access-g26dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:03:59.040710 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.040674    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g26dh\" (UniqueName: \"kubernetes.io/projected/4eab7e38-a04a-4690-b80a-ffe568a6e56e-kube-api-access-g26dh\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:03:59.040710 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.040704    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4eab7e38-a04a-4690-b80a-ffe568a6e56e-config-file\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:03:59.457866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.457827    2580 generic.go:356] "Generic (PLEG): container finished" podID="4eab7e38-a04a-4690-b80a-ffe568a6e56e" containerID="fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1" exitCode=0
Jul 07 18:03:59.458330 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.457878    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr"
Jul 07 18:03:59.458330 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.457939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr" event={"ID":"4eab7e38-a04a-4690-b80a-ffe568a6e56e","Type":"ContainerDied","Data":"fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1"}
Jul 07 18:03:59.458330 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.457993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-75kpr" event={"ID":"4eab7e38-a04a-4690-b80a-ffe568a6e56e","Type":"ContainerDied","Data":"3121047b1ae5975bab3c84d9ba5c9e50142279ac2208353e6cfe4c3738d5a463"}
Jul 07 18:03:59.458330 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.458018    2580 scope.go:117] "RemoveContainer" containerID="fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1"
Jul 07 18:03:59.468022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.468003    2580 scope.go:117] "RemoveContainer" containerID="fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1"
Jul 07 18:03:59.468333 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:03:59.468313    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1\": container with ID starting with fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1 not found: ID does not exist" containerID="fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1"
Jul 07 18:03:59.468379 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.468344    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1"} err="failed to get container status \"fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1\": rpc error: code = NotFound desc = could not find container \"fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1\": container with ID starting with fbb7a5e8b7f490c8f5a210eaa3994415f2a7b73d4f0e08441d3d93b406d2b8e1 not found: ID does not exist"
Jul 07 18:03:59.480838 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.480803    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:59.484702 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.484679    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-75kpr"]
Jul 07 18:03:59.990336 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990302    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-tspjk"]
Jul 07 18:03:59.990693 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990678    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f88307b0-95cd-4230-b378-8b00c0e157f9" containerName="authorino"
Jul 07 18:03:59.990760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990694    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f88307b0-95cd-4230-b378-8b00c0e157f9" containerName="authorino"
Jul 07 18:03:59.990760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990717    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eab7e38-a04a-4690-b80a-ffe568a6e56e" containerName="limitador"
Jul 07 18:03:59.990760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990722    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eab7e38-a04a-4690-b80a-ffe568a6e56e" containerName="limitador"
Jul 07 18:03:59.991029 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990780    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f88307b0-95cd-4230-b378-8b00c0e157f9" containerName="authorino"
Jul 07 18:03:59.991029 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.990789    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eab7e38-a04a-4690-b80a-ffe568a6e56e" containerName="limitador"
Jul 07 18:03:59.995252 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.995229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:03:59.998760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.998739    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 07 18:03:59.998995 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:03:59.998982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9xlxr\""
Jul 07 18:04:00.007150 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.007122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-tspjk"]
Jul 07 18:04:00.151210 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.151169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7d39e3d3-eb55-4174-8b36-93ed62758f73-data\") pod \"postgres-69fb4594cf-tspjk\" (UID: \"7d39e3d3-eb55-4174-8b36-93ed62758f73\") " pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.151401 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.151226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj94n\" (UniqueName: \"kubernetes.io/projected/7d39e3d3-eb55-4174-8b36-93ed62758f73-kube-api-access-mj94n\") pod \"postgres-69fb4594cf-tspjk\" (UID: \"7d39e3d3-eb55-4174-8b36-93ed62758f73\") " pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.252747 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.252654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7d39e3d3-eb55-4174-8b36-93ed62758f73-data\") pod \"postgres-69fb4594cf-tspjk\" (UID: \"7d39e3d3-eb55-4174-8b36-93ed62758f73\") " pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.252747 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.252721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj94n\" (UniqueName: \"kubernetes.io/projected/7d39e3d3-eb55-4174-8b36-93ed62758f73-kube-api-access-mj94n\") pod \"postgres-69fb4594cf-tspjk\" (UID: \"7d39e3d3-eb55-4174-8b36-93ed62758f73\") " pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.253117 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.253099    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7d39e3d3-eb55-4174-8b36-93ed62758f73-data\") pod \"postgres-69fb4594cf-tspjk\" (UID: \"7d39e3d3-eb55-4174-8b36-93ed62758f73\") " pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.262123 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.262097    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj94n\" (UniqueName: \"kubernetes.io/projected/7d39e3d3-eb55-4174-8b36-93ed62758f73-kube-api-access-mj94n\") pod \"postgres-69fb4594cf-tspjk\" (UID: \"7d39e3d3-eb55-4174-8b36-93ed62758f73\") " pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.307227 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.307193    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:00.446691 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.446666    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-tspjk"]
Jul 07 18:04:00.448375 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:00.448347    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d39e3d3_eb55_4174_8b36_93ed62758f73.slice/crio-122cb9d7ca2b8a4749653e42710456883d06cfc7217dd00bf91e827d7d39e778 WatchSource:0}: Error finding container 122cb9d7ca2b8a4749653e42710456883d06cfc7217dd00bf91e827d7d39e778: Status 404 returned error can't find the container with id 122cb9d7ca2b8a4749653e42710456883d06cfc7217dd00bf91e827d7d39e778
Jul 07 18:04:00.465869 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.465837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-tspjk" event={"ID":"7d39e3d3-eb55-4174-8b36-93ed62758f73","Type":"ContainerStarted","Data":"122cb9d7ca2b8a4749653e42710456883d06cfc7217dd00bf91e827d7d39e778"}
Jul 07 18:04:00.594767 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:00.594728    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eab7e38-a04a-4690-b80a-ffe568a6e56e" path="/var/lib/kubelet/pods/4eab7e38-a04a-4690-b80a-ffe568a6e56e/volumes"
Jul 07 18:04:06.504890 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:06.504841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-tspjk" event={"ID":"7d39e3d3-eb55-4174-8b36-93ed62758f73","Type":"ContainerStarted","Data":"317c9cc39e3c1a9b2c7c726d15fb0b3fc164daef6aa079939b7c3d55061805e1"}
Jul 07 18:04:06.505368 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:06.504961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:06.522549 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:06.522498    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-tspjk" podStartSLOduration=2.415918357 podStartE2EDuration="7.522481947s" totalImagesPullingTime="5.10656359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:04:06.521923005 +0000 UTC m=+608.412283725" watchObservedRunningTime="2026-07-07 18:04:06.522481947 +0000 UTC m=+608.412842671"
Jul 07 18:04:12.541369 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:12.541340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-tspjk"
Jul 07 18:04:13.148102 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.148058    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lzpkx"]
Jul 07 18:04:13.151300 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.151280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:13.154130 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.154111    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bdkx7\""
Jul 07 18:04:13.162046 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.162023    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lzpkx"]
Jul 07 18:04:13.165973 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.165949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdfjh\" (UniqueName: \"kubernetes.io/projected/b75349e6-8476-4bbd-af37-e27f6bbed2c0-kube-api-access-mdfjh\") pod \"authorino-5d4b67c446-lzpkx\" (UID: \"b75349e6-8476-4bbd-af37-e27f6bbed2c0\") " pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:13.266580 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.266546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdfjh\" (UniqueName: \"kubernetes.io/projected/b75349e6-8476-4bbd-af37-e27f6bbed2c0-kube-api-access-mdfjh\") pod \"authorino-5d4b67c446-lzpkx\" (UID: \"b75349e6-8476-4bbd-af37-e27f6bbed2c0\") " pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:13.278858 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.278816    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdfjh\" (UniqueName: \"kubernetes.io/projected/b75349e6-8476-4bbd-af37-e27f6bbed2c0-kube-api-access-mdfjh\") pod \"authorino-5d4b67c446-lzpkx\" (UID: \"b75349e6-8476-4bbd-af37-e27f6bbed2c0\") " pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:13.396298 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.396263    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lzpkx"]
Jul 07 18:04:13.396534 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.396522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:13.427451 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.427414    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5959957998-flrpk"]
Jul 07 18:04:13.432395 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.432366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:13.444403 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.442814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5959957998-flrpk"]
Jul 07 18:04:13.541799 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.541773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lzpkx"]
Jul 07 18:04:13.543935 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:13.543877    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb75349e6_8476_4bbd_af37_e27f6bbed2c0.slice/crio-0651fb299ebd2a3c5f8e6857117fa21df0ebdf273cd19de954c59ee84f6e84a8 WatchSource:0}: Error finding container 0651fb299ebd2a3c5f8e6857117fa21df0ebdf273cd19de954c59ee84f6e84a8: Status 404 returned error can't find the container with id 0651fb299ebd2a3c5f8e6857117fa21df0ebdf273cd19de954c59ee84f6e84a8
Jul 07 18:04:13.569365 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.569339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rldjn\" (UniqueName: \"kubernetes.io/projected/74f353ed-507b-491d-90c7-dfe4e5fc3042-kube-api-access-rldjn\") pod \"authorino-5959957998-flrpk\" (UID: \"74f353ed-507b-491d-90c7-dfe4e5fc3042\") " pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:13.669896 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.669809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rldjn\" (UniqueName: \"kubernetes.io/projected/74f353ed-507b-491d-90c7-dfe4e5fc3042-kube-api-access-rldjn\") pod \"authorino-5959957998-flrpk\" (UID: \"74f353ed-507b-491d-90c7-dfe4e5fc3042\") " pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:13.679872 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.679843    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rldjn\" (UniqueName: \"kubernetes.io/projected/74f353ed-507b-491d-90c7-dfe4e5fc3042-kube-api-access-rldjn\") pod \"authorino-5959957998-flrpk\" (UID: \"74f353ed-507b-491d-90c7-dfe4e5fc3042\") " pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:13.688097 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.688074    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5959957998-flrpk"]
Jul 07 18:04:13.688331 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.688320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:13.728988 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.728949    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57c8998946-f9nn4"]
Jul 07 18:04:13.734724 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.734697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:13.737725 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.737705    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 18:04:13.746412 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.746384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57c8998946-f9nn4"]
Jul 07 18:04:13.770663 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.770633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8xz\" (UniqueName: \"kubernetes.io/projected/9776a412-1bda-4fc7-80d6-359d90351c0e-kube-api-access-xw8xz\") pod \"authorino-57c8998946-f9nn4\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") " pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:13.770830 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.770681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9776a412-1bda-4fc7-80d6-359d90351c0e-tls-cert\") pod \"authorino-57c8998946-f9nn4\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") " pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:13.831804 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.831775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5959957998-flrpk"]
Jul 07 18:04:13.833218 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:13.833194    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74f353ed_507b_491d_90c7_dfe4e5fc3042.slice/crio-5ae9965594897776842185d6d48f9327fdc18409294bfe28c82ae31e9ce926bc WatchSource:0}: Error finding container 5ae9965594897776842185d6d48f9327fdc18409294bfe28c82ae31e9ce926bc: Status 404 returned error can't find the container with id 5ae9965594897776842185d6d48f9327fdc18409294bfe28c82ae31e9ce926bc
Jul 07 18:04:13.834663 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.834644    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 18:04:13.871634 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.871607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8xz\" (UniqueName: \"kubernetes.io/projected/9776a412-1bda-4fc7-80d6-359d90351c0e-kube-api-access-xw8xz\") pod \"authorino-57c8998946-f9nn4\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") " pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:13.871775 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.871655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9776a412-1bda-4fc7-80d6-359d90351c0e-tls-cert\") pod \"authorino-57c8998946-f9nn4\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") " pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:13.874165 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.874144    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9776a412-1bda-4fc7-80d6-359d90351c0e-tls-cert\") pod \"authorino-57c8998946-f9nn4\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") " pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:13.880512 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:13.880486    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8xz\" (UniqueName: \"kubernetes.io/projected/9776a412-1bda-4fc7-80d6-359d90351c0e-kube-api-access-xw8xz\") pod \"authorino-57c8998946-f9nn4\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") " pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:14.047371 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.047279    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:04:14.192838 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.192807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57c8998946-f9nn4"]
Jul 07 18:04:14.194425 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:14.194401    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9776a412_1bda_4fc7_80d6_359d90351c0e.slice/crio-85eabad48bb3a5d36bc502a40983d08be2ae79f476213c2a88635941ba9da542 WatchSource:0}: Error finding container 85eabad48bb3a5d36bc502a40983d08be2ae79f476213c2a88635941ba9da542: Status 404 returned error can't find the container with id 85eabad48bb3a5d36bc502a40983d08be2ae79f476213c2a88635941ba9da542
Jul 07 18:04:14.551502 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.551475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5959957998-flrpk" event={"ID":"74f353ed-507b-491d-90c7-dfe4e5fc3042","Type":"ContainerStarted","Data":"5ae9965594897776842185d6d48f9327fdc18409294bfe28c82ae31e9ce926bc"}
Jul 07 18:04:14.552972 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.552950    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lzpkx" event={"ID":"b75349e6-8476-4bbd-af37-e27f6bbed2c0","Type":"ContainerStarted","Data":"59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c"}
Jul 07 18:04:14.553088 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.552984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lzpkx" event={"ID":"b75349e6-8476-4bbd-af37-e27f6bbed2c0","Type":"ContainerStarted","Data":"0651fb299ebd2a3c5f8e6857117fa21df0ebdf273cd19de954c59ee84f6e84a8"}
Jul 07 18:04:14.553088 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.553004    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-lzpkx" podUID="b75349e6-8476-4bbd-af37-e27f6bbed2c0" containerName="authorino" containerID="cri-o://59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c" gracePeriod=30
Jul 07 18:04:14.554347 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.554325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c8998946-f9nn4" event={"ID":"9776a412-1bda-4fc7-80d6-359d90351c0e","Type":"ContainerStarted","Data":"85eabad48bb3a5d36bc502a40983d08be2ae79f476213c2a88635941ba9da542"}
Jul 07 18:04:14.577375 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.577322    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-lzpkx" podStartSLOduration=0.811587778 podStartE2EDuration="1.577305917s" totalImagesPullingTime="765.718139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:04:14.574772896 +0000 UTC m=+616.465133641" watchObservedRunningTime="2026-07-07 18:04:14.577305917 +0000 UTC m=+616.467666642"
Jul 07 18:04:14.824321 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.824264    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:14.879546 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.879511    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdfjh\" (UniqueName: \"kubernetes.io/projected/b75349e6-8476-4bbd-af37-e27f6bbed2c0-kube-api-access-mdfjh\") pod \"b75349e6-8476-4bbd-af37-e27f6bbed2c0\" (UID: \"b75349e6-8476-4bbd-af37-e27f6bbed2c0\") "
Jul 07 18:04:14.881668 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.881643    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b75349e6-8476-4bbd-af37-e27f6bbed2c0-kube-api-access-mdfjh" (OuterVolumeSpecName: "kube-api-access-mdfjh") pod "b75349e6-8476-4bbd-af37-e27f6bbed2c0" (UID: "b75349e6-8476-4bbd-af37-e27f6bbed2c0"). InnerVolumeSpecName "kube-api-access-mdfjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:04:14.980859 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:14.980819    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdfjh\" (UniqueName: \"kubernetes.io/projected/b75349e6-8476-4bbd-af37-e27f6bbed2c0-kube-api-access-mdfjh\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:04:15.561077 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.560979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c8998946-f9nn4" event={"ID":"9776a412-1bda-4fc7-80d6-359d90351c0e","Type":"ContainerStarted","Data":"8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1"}
Jul 07 18:04:15.562473 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.562444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5959957998-flrpk" event={"ID":"74f353ed-507b-491d-90c7-dfe4e5fc3042","Type":"ContainerStarted","Data":"589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe"}
Jul 07 18:04:15.562612 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.562519    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5959957998-flrpk" podUID="74f353ed-507b-491d-90c7-dfe4e5fc3042" containerName="authorino" containerID="cri-o://589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe" gracePeriod=30
Jul 07 18:04:15.563792 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.563763    2580 generic.go:356] "Generic (PLEG): container finished" podID="b75349e6-8476-4bbd-af37-e27f6bbed2c0" containerID="59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c" exitCode=0
Jul 07 18:04:15.563860 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.563807    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lzpkx"
Jul 07 18:04:15.563928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.563808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lzpkx" event={"ID":"b75349e6-8476-4bbd-af37-e27f6bbed2c0","Type":"ContainerDied","Data":"59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c"}
Jul 07 18:04:15.563928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.563922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lzpkx" event={"ID":"b75349e6-8476-4bbd-af37-e27f6bbed2c0","Type":"ContainerDied","Data":"0651fb299ebd2a3c5f8e6857117fa21df0ebdf273cd19de954c59ee84f6e84a8"}
Jul 07 18:04:15.564015 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.563944    2580 scope.go:117] "RemoveContainer" containerID="59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c"
Jul 07 18:04:15.574201 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.574184    2580 scope.go:117] "RemoveContainer" containerID="59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c"
Jul 07 18:04:15.574487 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:04:15.574471    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c\": container with ID starting with 59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c not found: ID does not exist" containerID="59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c"
Jul 07 18:04:15.574536 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.574496    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c"} err="failed to get container status \"59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c\": rpc error: code = NotFound desc = could not find container \"59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c\": container with ID starting with 59160a2f1c895dbbbf11bbfb2161599da75c5343ebf3448409e8e5ea6e65d85c not found: ID does not exist"
Jul 07 18:04:15.580147 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.580095    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57c8998946-f9nn4" podStartSLOduration=2.171906223 podStartE2EDuration="2.580083072s" totalImagesPullingTime="408.176849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:04:15.578717156 +0000 UTC m=+617.469077880" watchObservedRunningTime="2026-07-07 18:04:15.580083072 +0000 UTC m=+617.470443773"
Jul 07 18:04:15.596458 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.596411    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5959957998-flrpk" podStartSLOduration=1.903993008 podStartE2EDuration="2.596395656s" totalImagesPullingTime="692.402648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:04:15.594100932 +0000 UTC m=+617.484461657" watchObservedRunningTime="2026-07-07 18:04:15.596395656 +0000 UTC m=+617.486756379"
Jul 07 18:04:15.613739 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.613715    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lzpkx"]
Jul 07 18:04:15.617737 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.617703    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lzpkx"]
Jul 07 18:04:15.826818 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.826758    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:15.888845 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.888814    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rldjn\" (UniqueName: \"kubernetes.io/projected/74f353ed-507b-491d-90c7-dfe4e5fc3042-kube-api-access-rldjn\") pod \"74f353ed-507b-491d-90c7-dfe4e5fc3042\" (UID: \"74f353ed-507b-491d-90c7-dfe4e5fc3042\") "
Jul 07 18:04:15.891083 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.891054    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74f353ed-507b-491d-90c7-dfe4e5fc3042-kube-api-access-rldjn" (OuterVolumeSpecName: "kube-api-access-rldjn") pod "74f353ed-507b-491d-90c7-dfe4e5fc3042" (UID: "74f353ed-507b-491d-90c7-dfe4e5fc3042"). InnerVolumeSpecName "kube-api-access-rldjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:04:15.989878 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:15.989841    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rldjn\" (UniqueName: \"kubernetes.io/projected/74f353ed-507b-491d-90c7-dfe4e5fc3042-kube-api-access-rldjn\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:04:16.570095 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.570062    2580 generic.go:356] "Generic (PLEG): container finished" podID="74f353ed-507b-491d-90c7-dfe4e5fc3042" containerID="589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe" exitCode=0
Jul 07 18:04:16.570542 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.570101    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5959957998-flrpk"
Jul 07 18:04:16.570542 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.570139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5959957998-flrpk" event={"ID":"74f353ed-507b-491d-90c7-dfe4e5fc3042","Type":"ContainerDied","Data":"589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe"}
Jul 07 18:04:16.570542 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.570183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5959957998-flrpk" event={"ID":"74f353ed-507b-491d-90c7-dfe4e5fc3042","Type":"ContainerDied","Data":"5ae9965594897776842185d6d48f9327fdc18409294bfe28c82ae31e9ce926bc"}
Jul 07 18:04:16.570542 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.570201    2580 scope.go:117] "RemoveContainer" containerID="589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe"
Jul 07 18:04:16.579689 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.579671    2580 scope.go:117] "RemoveContainer" containerID="589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe"
Jul 07 18:04:16.579963 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:04:16.579944    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe\": container with ID starting with 589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe not found: ID does not exist" containerID="589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe"
Jul 07 18:04:16.580020 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.579974    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe"} err="failed to get container status \"589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe\": rpc error: code = NotFound desc = could not find container \"589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe\": container with ID starting with 589c79550647808080a861be1701ce4740ee81379a340349a06a3d9956f950fe not found: ID does not exist"
Jul 07 18:04:16.597018 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.596985    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b75349e6-8476-4bbd-af37-e27f6bbed2c0" path="/var/lib/kubelet/pods/b75349e6-8476-4bbd-af37-e27f6bbed2c0/volumes"
Jul 07 18:04:16.597372 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.597356    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5959957998-flrpk"]
Jul 07 18:04:16.597788 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:16.597773    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5959957998-flrpk"]
Jul 07 18:04:18.594972 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:18.594856    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74f353ed-507b-491d-90c7-dfe4e5fc3042" path="/var/lib/kubelet/pods/74f353ed-507b-491d-90c7-dfe4e5fc3042/volumes"
Jul 07 18:04:20.280990 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.280951    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bc8df45b5-pxzjn"]
Jul 07 18:04:20.281414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.281375    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75349e6-8476-4bbd-af37-e27f6bbed2c0" containerName="authorino"
Jul 07 18:04:20.281414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.281387    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75349e6-8476-4bbd-af37-e27f6bbed2c0" containerName="authorino"
Jul 07 18:04:20.281414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.281402    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74f353ed-507b-491d-90c7-dfe4e5fc3042" containerName="authorino"
Jul 07 18:04:20.281414 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.281410    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="74f353ed-507b-491d-90c7-dfe4e5fc3042" containerName="authorino"
Jul 07 18:04:20.281593 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.281497    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="74f353ed-507b-491d-90c7-dfe4e5fc3042" containerName="authorino"
Jul 07 18:04:20.281593 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.281509    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75349e6-8476-4bbd-af37-e27f6bbed2c0" containerName="authorino"
Jul 07 18:04:20.511928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.511862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bc8df45b5-pxzjn"]
Jul 07 18:04:20.511928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.511913    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5594d854cf-2hgvl"]
Jul 07 18:04:20.512198 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.512019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.516005 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.515863    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 18:04:20.516005 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.515888    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 18:04:20.516005 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.515863    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tm82w\""
Jul 07 18:04:20.522795 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.522771    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5594d854cf-2hgvl"]
Jul 07 18:04:20.522937 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.522883    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.530750 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.530721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh6g7\" (UniqueName: \"kubernetes.io/projected/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-kube-api-access-rh6g7\") pod \"maas-controller-6bc8df45b5-pxzjn\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") " pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.530942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.530758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-webhook-cert\") pod \"maas-controller-6bc8df45b5-pxzjn\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") " pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.631749 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.631714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed7cf592-2a20-4378-af9d-ba58422c2b8c-webhook-cert\") pod \"maas-controller-5594d854cf-2hgvl\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") " pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.631969 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.631790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh6g7\" (UniqueName: \"kubernetes.io/projected/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-kube-api-access-rh6g7\") pod \"maas-controller-6bc8df45b5-pxzjn\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") " pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.631969 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.631825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-webhook-cert\") pod \"maas-controller-6bc8df45b5-pxzjn\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") " pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.631969 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.631842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrnbj\" (UniqueName: \"kubernetes.io/projected/ed7cf592-2a20-4378-af9d-ba58422c2b8c-kube-api-access-wrnbj\") pod \"maas-controller-5594d854cf-2hgvl\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") " pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.634482 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.634459    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-webhook-cert\") pod \"maas-controller-6bc8df45b5-pxzjn\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") " pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.642549 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.642436    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh6g7\" (UniqueName: \"kubernetes.io/projected/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-kube-api-access-rh6g7\") pod \"maas-controller-6bc8df45b5-pxzjn\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") " pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.732786 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.732746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrnbj\" (UniqueName: \"kubernetes.io/projected/ed7cf592-2a20-4378-af9d-ba58422c2b8c-kube-api-access-wrnbj\") pod \"maas-controller-5594d854cf-2hgvl\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") " pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.733012 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.732820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed7cf592-2a20-4378-af9d-ba58422c2b8c-webhook-cert\") pod \"maas-controller-5594d854cf-2hgvl\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") " pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.735446 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.735428    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed7cf592-2a20-4378-af9d-ba58422c2b8c-webhook-cert\") pod \"maas-controller-5594d854cf-2hgvl\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") " pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.748023 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.747995    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrnbj\" (UniqueName: \"kubernetes.io/projected/ed7cf592-2a20-4378-af9d-ba58422c2b8c-kube-api-access-wrnbj\") pod \"maas-controller-5594d854cf-2hgvl\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") " pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.823281 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.823247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:20.832551 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.832528    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:20.995816 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:20.995785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bc8df45b5-pxzjn"]
Jul 07 18:04:20.997087 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:20.997061    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a8b51c_d226_4a3f_bfe0_ae5672a69f8e.slice/crio-5a00eb88708637f69aba3bd4d2a3e77a2d880333f0b51afb10971d99d281829f WatchSource:0}: Error finding container 5a00eb88708637f69aba3bd4d2a3e77a2d880333f0b51afb10971d99d281829f: Status 404 returned error can't find the container with id 5a00eb88708637f69aba3bd4d2a3e77a2d880333f0b51afb10971d99d281829f
Jul 07 18:04:21.029433 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:21.029403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5594d854cf-2hgvl"]
Jul 07 18:04:21.030796 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:21.030771    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7cf592_2a20_4378_af9d_ba58422c2b8c.slice/crio-2ad887c1f0f0415a47c78f4ed3052ff825d0384cce3b6a0b329de6c1fe48bfcd WatchSource:0}: Error finding container 2ad887c1f0f0415a47c78f4ed3052ff825d0384cce3b6a0b329de6c1fe48bfcd: Status 404 returned error can't find the container with id 2ad887c1f0f0415a47c78f4ed3052ff825d0384cce3b6a0b329de6c1fe48bfcd
Jul 07 18:04:21.603227 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:21.603191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5594d854cf-2hgvl" event={"ID":"ed7cf592-2a20-4378-af9d-ba58422c2b8c","Type":"ContainerStarted","Data":"2ad887c1f0f0415a47c78f4ed3052ff825d0384cce3b6a0b329de6c1fe48bfcd"}
Jul 07 18:04:21.604385 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:21.604365    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn" event={"ID":"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e","Type":"ContainerStarted","Data":"5a00eb88708637f69aba3bd4d2a3e77a2d880333f0b51afb10971d99d281829f"}
Jul 07 18:04:23.619646 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:23.619591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5594d854cf-2hgvl" event={"ID":"ed7cf592-2a20-4378-af9d-ba58422c2b8c","Type":"ContainerStarted","Data":"3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419"}
Jul 07 18:04:23.620190 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:23.619743    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:23.621551 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:23.621522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn" event={"ID":"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e","Type":"ContainerStarted","Data":"6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75"}
Jul 07 18:04:23.621656 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:23.621579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:23.667797 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:23.667733    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5594d854cf-2hgvl" podStartSLOduration=1.360083191 podStartE2EDuration="3.667717106s" totalImagesPullingTime="2.307633915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:04:23.663979017 +0000 UTC m=+625.554339741" watchObservedRunningTime="2026-07-07 18:04:23.667717106 +0000 UTC m=+625.558077829"
Jul 07 18:04:23.686372 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:23.686306    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn" podStartSLOduration=1.3583497580000001 podStartE2EDuration="3.68628554s" totalImagesPullingTime="2.327935782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:04:23.685285612 +0000 UTC m=+625.575646362" watchObservedRunningTime="2026-07-07 18:04:23.68628554 +0000 UTC m=+625.576646265"
Jul 07 18:04:34.638694 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:34.638659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:34.639159 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:34.638883    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:04:34.706452 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:34.706422    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bc8df45b5-pxzjn"]
Jul 07 18:04:34.706673 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:34.706646    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn" podUID="a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" containerName="manager" containerID="cri-o://6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75" gracePeriod=30
Jul 07 18:04:34.958695 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:34.958674    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:35.063706 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.063670    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-webhook-cert\") pod \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") "
Jul 07 18:04:35.063893 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.063770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh6g7\" (UniqueName: \"kubernetes.io/projected/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-kube-api-access-rh6g7\") pod \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\" (UID: \"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e\") "
Jul 07 18:04:35.066103 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.066069    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" (UID: "a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:04:35.066231 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.066094    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-kube-api-access-rh6g7" (OuterVolumeSpecName: "kube-api-access-rh6g7") pod "a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" (UID: "a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e"). InnerVolumeSpecName "kube-api-access-rh6g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:04:35.165017 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.164976    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh6g7\" (UniqueName: \"kubernetes.io/projected/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-kube-api-access-rh6g7\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:04:35.165017 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.165009    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e-webhook-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:04:35.689663 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.689623    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" containerID="6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75" exitCode=0
Jul 07 18:04:35.689663 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.689664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn" event={"ID":"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e","Type":"ContainerDied","Data":"6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75"}
Jul 07 18:04:35.690243 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.689671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn"
Jul 07 18:04:35.690243 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.689713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bc8df45b5-pxzjn" event={"ID":"a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e","Type":"ContainerDied","Data":"5a00eb88708637f69aba3bd4d2a3e77a2d880333f0b51afb10971d99d281829f"}
Jul 07 18:04:35.690243 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.689747    2580 scope.go:117] "RemoveContainer" containerID="6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75"
Jul 07 18:04:35.705756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.705732    2580 scope.go:117] "RemoveContainer" containerID="6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75"
Jul 07 18:04:35.706056 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:04:35.706036    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75\": container with ID starting with 6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75 not found: ID does not exist" containerID="6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75"
Jul 07 18:04:35.706121 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.706065    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75"} err="failed to get container status \"6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75\": rpc error: code = NotFound desc = could not find container \"6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75\": container with ID starting with 6b338646da28d66a66debc3d728c7f09322bcfb1f64c708cb1737e11e22c0b75 not found: ID does not exist"
Jul 07 18:04:35.719107 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.719072    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bc8df45b5-pxzjn"]
Jul 07 18:04:35.721348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:35.721326    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bc8df45b5-pxzjn"]
Jul 07 18:04:36.595924 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:36.595876    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" path="/var/lib/kubelet/pods/a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e/volumes"
Jul 07 18:04:57.854595 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.854550    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-66f9f5f9bc-fxh7n"]
Jul 07 18:04:57.855298 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.855173    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" containerName="manager"
Jul 07 18:04:57.855298 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.855194    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" containerName="manager"
Jul 07 18:04:57.855298 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.855295    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a8b51c-d226-4a3f-bfe0-ae5672a69f8e" containerName="manager"
Jul 07 18:04:57.858417 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.858395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.863378 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.863357    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-jnv2d\""
Jul 07 18:04:57.863586 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.863573    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 07 18:04:57.868688 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.868395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-openshift-service-ca\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.868688 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.868540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-maas-api-tls\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.868688 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.868577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4hhz\" (UniqueName: \"kubernetes.io/projected/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-kube-api-access-c4hhz\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.871478 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.871454    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-66f9f5f9bc-fxh7n"]
Jul 07 18:04:57.969084 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.969044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-maas-api-tls\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.969293 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.969095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4hhz\" (UniqueName: \"kubernetes.io/projected/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-kube-api-access-c4hhz\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.969293 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.969133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-openshift-service-ca\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.969293 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:04:57.969222    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 07 18:04:57.969471 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:04:57.969319    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-maas-api-tls podName:7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b nodeName:}" failed. No retries permitted until 2026-07-07 18:04:58.469296544 +0000 UTC m=+660.359657253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-maas-api-tls") pod "maas-api-66f9f5f9bc-fxh7n" (UID: "7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b") : secret "maas-api-serving-cert" not found
Jul 07 18:04:57.969823 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.969807    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-openshift-service-ca\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:57.979596 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:57.979574    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4hhz\" (UniqueName: \"kubernetes.io/projected/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-kube-api-access-c4hhz\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:58.440284 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.440249    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5s7q8"]
Jul 07 18:04:58.442895 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.442873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.447264 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.447235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tlz7q\""
Jul 07 18:04:58.447554 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.447536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 18:04:58.457189 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.457164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5s7q8"]
Jul 07 18:04:58.473293 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.473264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-maas-api-tls\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:58.473472 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.473304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcpf2\" (UniqueName: \"kubernetes.io/projected/c1186ff2-608f-40c6-8a19-50eee9cfbf78-kube-api-access-bcpf2\") pod \"payload-processing-58f86c7798-5s7q8\" (UID: \"c1186ff2-608f-40c6-8a19-50eee9cfbf78\") " pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.473472 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.473355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1186ff2-608f-40c6-8a19-50eee9cfbf78-plugins-config-volume\") pod \"payload-processing-58f86c7798-5s7q8\" (UID: \"c1186ff2-608f-40c6-8a19-50eee9cfbf78\") " pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.475982 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.475955    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b-maas-api-tls\") pod \"maas-api-66f9f5f9bc-fxh7n\" (UID: \"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b\") " pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:58.490056 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.490026    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kkxss"]
Jul 07 18:04:58.493128 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.493113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.504212 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.504187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kkxss"]
Jul 07 18:04:58.574031 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.574012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1186ff2-608f-40c6-8a19-50eee9cfbf78-plugins-config-volume\") pod \"payload-processing-58f86c7798-5s7q8\" (UID: \"c1186ff2-608f-40c6-8a19-50eee9cfbf78\") " pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.574118 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.574070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcpf2\" (UniqueName: \"kubernetes.io/projected/c1186ff2-608f-40c6-8a19-50eee9cfbf78-kube-api-access-bcpf2\") pod \"payload-processing-58f86c7798-5s7q8\" (UID: \"c1186ff2-608f-40c6-8a19-50eee9cfbf78\") " pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.574118 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.574096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebf4b7a-afe6-438c-8861-8ee027bf506b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kkxss\" (UID: \"bebf4b7a-afe6-438c-8861-8ee027bf506b\") " pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.574118 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.574114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x59xr\" (UniqueName: \"kubernetes.io/projected/bebf4b7a-afe6-438c-8861-8ee027bf506b-kube-api-access-x59xr\") pod \"payload-pre-processing-7964457696-kkxss\" (UID: \"bebf4b7a-afe6-438c-8861-8ee027bf506b\") " pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.574735 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.574711    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1186ff2-608f-40c6-8a19-50eee9cfbf78-plugins-config-volume\") pod \"payload-processing-58f86c7798-5s7q8\" (UID: \"c1186ff2-608f-40c6-8a19-50eee9cfbf78\") " pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.582604 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.582580    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcpf2\" (UniqueName: \"kubernetes.io/projected/c1186ff2-608f-40c6-8a19-50eee9cfbf78-kube-api-access-bcpf2\") pod \"payload-processing-58f86c7798-5s7q8\" (UID: \"c1186ff2-608f-40c6-8a19-50eee9cfbf78\") " pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.674566 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.674519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebf4b7a-afe6-438c-8861-8ee027bf506b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kkxss\" (UID: \"bebf4b7a-afe6-438c-8861-8ee027bf506b\") " pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.674566 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.674558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x59xr\" (UniqueName: \"kubernetes.io/projected/bebf4b7a-afe6-438c-8861-8ee027bf506b-kube-api-access-x59xr\") pod \"payload-pre-processing-7964457696-kkxss\" (UID: \"bebf4b7a-afe6-438c-8861-8ee027bf506b\") " pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.675519 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.675491    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bebf4b7a-afe6-438c-8861-8ee027bf506b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kkxss\" (UID: \"bebf4b7a-afe6-438c-8861-8ee027bf506b\") " pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.685052 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.685020    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x59xr\" (UniqueName: \"kubernetes.io/projected/bebf4b7a-afe6-438c-8861-8ee027bf506b-kube-api-access-x59xr\") pod \"payload-pre-processing-7964457696-kkxss\" (UID: \"bebf4b7a-afe6-438c-8861-8ee027bf506b\") " pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.754494 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.754409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:04:58.772855 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.772831    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-jnv2d\""
Jul 07 18:04:58.780319 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.780288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:04:58.803968 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.803897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:04:58.946916 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.946834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5s7q8"]
Jul 07 18:04:58.947329 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:58.947146    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1186ff2_608f_40c6_8a19_50eee9cfbf78.slice/crio-eaadb42eb19e3ab3dbc61f6416fee2fd65e6e7df28abfb42e8cd517ec2cfa8ac WatchSource:0}: Error finding container eaadb42eb19e3ab3dbc61f6416fee2fd65e6e7df28abfb42e8cd517ec2cfa8ac: Status 404 returned error can't find the container with id eaadb42eb19e3ab3dbc61f6416fee2fd65e6e7df28abfb42e8cd517ec2cfa8ac
Jul 07 18:04:58.999168 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:58.999137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-66f9f5f9bc-fxh7n"]
Jul 07 18:04:59.000984 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:59.000952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbef1fe_4bf7_4ac0_a814_ce21aeb0990b.slice/crio-9ed7894d6840f9528ccdefbf3e46bd513bce15648fa8f741266a1a389d3e5dc0 WatchSource:0}: Error finding container 9ed7894d6840f9528ccdefbf3e46bd513bce15648fa8f741266a1a389d3e5dc0: Status 404 returned error can't find the container with id 9ed7894d6840f9528ccdefbf3e46bd513bce15648fa8f741266a1a389d3e5dc0
Jul 07 18:04:59.027121 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:59.027094    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kkxss"]
Jul 07 18:04:59.028196 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:04:59.028174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebf4b7a_afe6_438c_8861_8ee027bf506b.slice/crio-6f92b16f483ce5fb83e639fdfdf20fa51a27e47c9a6c0ebe199fdb4e36c6419e WatchSource:0}: Error finding container 6f92b16f483ce5fb83e639fdfdf20fa51a27e47c9a6c0ebe199fdb4e36c6419e: Status 404 returned error can't find the container with id 6f92b16f483ce5fb83e639fdfdf20fa51a27e47c9a6c0ebe199fdb4e36c6419e
Jul 07 18:04:59.842246 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:59.842202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss" event={"ID":"bebf4b7a-afe6-438c-8861-8ee027bf506b","Type":"ContainerStarted","Data":"6f92b16f483ce5fb83e639fdfdf20fa51a27e47c9a6c0ebe199fdb4e36c6419e"}
Jul 07 18:04:59.844402 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:59.844369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8" event={"ID":"c1186ff2-608f-40c6-8a19-50eee9cfbf78","Type":"ContainerStarted","Data":"eaadb42eb19e3ab3dbc61f6416fee2fd65e6e7df28abfb42e8cd517ec2cfa8ac"}
Jul 07 18:04:59.848018 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:04:59.847989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n" event={"ID":"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b","Type":"ContainerStarted","Data":"9ed7894d6840f9528ccdefbf3e46bd513bce15648fa8f741266a1a389d3e5dc0"}
Jul 07 18:05:01.862555 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.862523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss" event={"ID":"bebf4b7a-afe6-438c-8861-8ee027bf506b","Type":"ContainerStarted","Data":"74a8017117a3e7edb8fc1726a198535dfc5e7aa4fff3ba7ddb284f9ff48b4de9"}
Jul 07 18:05:01.863060 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.862649    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:05:01.864212 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.864188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8" event={"ID":"c1186ff2-608f-40c6-8a19-50eee9cfbf78","Type":"ContainerStarted","Data":"76ceb17486c2c5400ba0350eee9924ec4b906c06d640f13e6d753e09af90571f"}
Jul 07 18:05:01.864320 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.864238    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:05:01.865731 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.865706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n" event={"ID":"7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b","Type":"ContainerStarted","Data":"83b7ae381b19958eee3220e1ab4d988a4b07daae32656658b39c3eda8153c101"}
Jul 07 18:05:01.865826 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.865758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:05:01.882802 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.882714    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss" podStartSLOduration=1.66261972 podStartE2EDuration="3.882697772s" totalImagesPullingTime="2.220078052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:05:01.881464672 +0000 UTC m=+663.771825397" watchObservedRunningTime="2026-07-07 18:05:01.882697772 +0000 UTC m=+663.773058545"
Jul 07 18:05:01.901113 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.901016    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n" podStartSLOduration=2.655569384 podStartE2EDuration="4.901000048s" totalImagesPullingTime="2.245430664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:05:01.89948457 +0000 UTC m=+663.789845292" watchObservedRunningTime="2026-07-07 18:05:01.901000048 +0000 UTC m=+663.791360770"
Jul 07 18:05:01.924130 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:01.924064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8" podStartSLOduration=1.6260120630000001 podStartE2EDuration="3.924038638s" totalImagesPullingTime="2.298026575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:04:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:05:01.919680392 +0000 UTC m=+663.810041115" watchObservedRunningTime="2026-07-07 18:05:01.924038638 +0000 UTC m=+663.814399362"
Jul 07 18:05:06.898632 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:06.898598    2580 generic.go:356] "Generic (PLEG): container finished" podID="422134db-9391-4170-a04a-613ea85b8ef0" containerID="a9a48f16c8194e5049978561f249f83aadcb9804ff5680d41c408d74ed4096db" exitCode=0
Jul 07 18:05:06.899146 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:06.898674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" event={"ID":"422134db-9391-4170-a04a-613ea85b8ef0","Type":"ContainerDied","Data":"a9a48f16c8194e5049978561f249f83aadcb9804ff5680d41c408d74ed4096db"}
Jul 07 18:05:06.899146 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:06.899116    2580 scope.go:117] "RemoveContainer" containerID="a9a48f16c8194e5049978561f249f83aadcb9804ff5680d41c408d74ed4096db"
Jul 07 18:05:06.900107 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:06.900074    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:05:06.900196 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:06.900146    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:05:06.900196 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:06.900176    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 07 18:05:07.877418 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:07.877389    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-66f9f5f9bc-fxh7n"
Jul 07 18:05:07.907594 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:07.907547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" event={"ID":"422134db-9391-4170-a04a-613ea85b8ef0","Type":"ContainerStarted","Data":"de8b942fa97d3cea59dd3b44cc5a9b8af15930ac1d03f1f8eed9c18bf3f9b997"}
Jul 07 18:05:08.370974 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:08.370939    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:05:08.371233 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:08.371211    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-49h8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 18:05:08.371285 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:08.371261    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" podUID="422134db-9391-4170-a04a-613ea85b8ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 18:05:09.371218 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:09.371179    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-49h8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 18:05:09.371643 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:09.371235    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" podUID="422134db-9391-4170-a04a-613ea85b8ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 18:05:10.370989 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:10.370948    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-49h8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 18:05:10.371165 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:10.371016    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" podUID="422134db-9391-4170-a04a-613ea85b8ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 18:05:11.371021 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:11.370980    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-49h8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 18:05:11.371411 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:11.371039    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f" podUID="422134db-9391-4170-a04a-613ea85b8ef0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 18:05:12.374751 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:12.374722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:05:12.375206 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:12.375182    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:05:12.375811 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:12.375791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-49h8f"
Jul 07 18:05:12.874039 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:12.874011    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5s7q8"
Jul 07 18:05:12.874258 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:12.874090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-kkxss"
Jul 07 18:05:17.985244 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.985209    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"]
Jul 07 18:05:17.989685 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.989660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:17.992709 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.992681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 18:05:17.992709 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.992677    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-45zl4\""
Jul 07 18:05:17.993562 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.993537    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 18:05:17.993679 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.993617    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 07 18:05:17.999045 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:17.999017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"]
Jul 07 18:05:18.069118 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.069084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39da832-7f53-4b8d-ae4b-d0564361f0f6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.069305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.069134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.069305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.069163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.069305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.069195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.069305 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.069231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.069447 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.069317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5jt2\" (UniqueName: \"kubernetes.io/projected/b39da832-7f53-4b8d-ae4b-d0564361f0f6-kube-api-access-x5jt2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170367 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5jt2\" (UniqueName: \"kubernetes.io/projected/b39da832-7f53-4b8d-ae4b-d0564361f0f6-kube-api-access-x5jt2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39da832-7f53-4b8d-ae4b-d0564361f0f6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170854 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170771    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170854 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170830    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.170980 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.170891    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.172946 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.172921    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39da832-7f53-4b8d-ae4b-d0564361f0f6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.173128 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.173111    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39da832-7f53-4b8d-ae4b-d0564361f0f6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.179455 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.179432    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5jt2\" (UniqueName: \"kubernetes.io/projected/b39da832-7f53-4b8d-ae4b-d0564361f0f6-kube-api-access-x5jt2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg\" (UID: \"b39da832-7f53-4b8d-ae4b-d0564361f0f6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.301577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.301483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:18.448063 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.448031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"]
Jul 07 18:05:18.449640 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:05:18.449617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39da832_7f53_4b8d_ae4b_d0564361f0f6.slice/crio-3b9b34a93cf8c9909bea7c9ce9b34d1eff5ed7430deb4e2f6aed071b3b702eeb WatchSource:0}: Error finding container 3b9b34a93cf8c9909bea7c9ce9b34d1eff5ed7430deb4e2f6aed071b3b702eeb: Status 404 returned error can't find the container with id 3b9b34a93cf8c9909bea7c9ce9b34d1eff5ed7430deb4e2f6aed071b3b702eeb
Jul 07 18:05:18.986160 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:18.986078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg" event={"ID":"b39da832-7f53-4b8d-ae4b-d0564361f0f6","Type":"ContainerStarted","Data":"3b9b34a93cf8c9909bea7c9ce9b34d1eff5ed7430deb4e2f6aed071b3b702eeb"}
Jul 07 18:05:19.994153 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:19.994053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg" event={"ID":"b39da832-7f53-4b8d-ae4b-d0564361f0f6","Type":"ContainerStarted","Data":"0cdfc3e2676af0b18047cc84f10994a5ae83c8f748e87d9ba0f05634482f4546"}
Jul 07 18:05:20.018915 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:20.018848    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg" podStartSLOduration=1.789951786 podStartE2EDuration="3.018832367s" totalImagesPullingTime="1.228880581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:05:20.016591991 +0000 UTC m=+681.906952716" watchObservedRunningTime="2026-07-07 18:05:20.018832367 +0000 UTC m=+681.909193091"
Jul 07 18:05:24.877475 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:24.877441    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"]
Jul 07 18:05:24.880189 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:24.880174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:24.883130 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:24.883112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 07 18:05:24.891942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:24.891894    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"]
Jul 07 18:05:25.037165 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.037129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd6bh\" (UniqueName: \"kubernetes.io/projected/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-kube-api-access-wd6bh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.037165 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.037164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.037384 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.037206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.037384 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.037267    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.037384 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.037281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.037384 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.037299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.138598 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.138515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.138598 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.138583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.138833 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.138642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.138833 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.138667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.138833 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.138695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.138833 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.138775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd6bh\" (UniqueName: \"kubernetes.io/projected/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-kube-api-access-wd6bh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.139071 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.139034    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.139135 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.139119    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.139214 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.139198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.141097 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.141065    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.141257 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.141241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.149614 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.149588    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd6bh\" (UniqueName: \"kubernetes.io/projected/e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f-kube-api-access-wd6bh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kjgq9\" (UID: \"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.191415 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.191384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:25.335033 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:25.335001    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"]
Jul 07 18:05:25.336699 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:05:25.336673    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode356bdf5_5f2c_45f3_a0af_d8c78ccbd87f.slice/crio-7eabbb3bf0612ae2b8a284e2400747f5875a0992ea2796e7b50b94f9cd602858 WatchSource:0}: Error finding container 7eabbb3bf0612ae2b8a284e2400747f5875a0992ea2796e7b50b94f9cd602858: Status 404 returned error can't find the container with id 7eabbb3bf0612ae2b8a284e2400747f5875a0992ea2796e7b50b94f9cd602858
Jul 07 18:05:26.032200 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:26.032111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9" event={"ID":"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f","Type":"ContainerStarted","Data":"a0e1cff3e26be9609c9e3cb3340678d2125867edb5f37772e287e506f9a30d15"}
Jul 07 18:05:26.032200 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:26.032170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9" event={"ID":"e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f","Type":"ContainerStarted","Data":"7eabbb3bf0612ae2b8a284e2400747f5875a0992ea2796e7b50b94f9cd602858"}
Jul 07 18:05:26.060980 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:26.060890    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9" podStartSLOduration=1.798753778 podStartE2EDuration="2.060868849s" totalImagesPullingTime="262.115071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:05:26.058997089 +0000 UTC m=+687.949357813" watchObservedRunningTime="2026-07-07 18:05:26.060868849 +0000 UTC m=+687.951229574"
Jul 07 18:05:28.301884 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:28.301838    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:28.301884 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:28.301900    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:28.314666 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:28.314636    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:29.059433 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:29.059401    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg"
Jul 07 18:05:35.192268 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:35.192224    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:35.192670 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:35.192301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:35.204880 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:35.204849    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:05:36.099272 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:05:36.099242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kjgq9"
Jul 07 18:06:08.388431 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.388334    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"]
Jul 07 18:06:08.392730 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.392710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.396089 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.396072    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 07 18:06:08.405497 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.405474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"]
Jul 07 18:06:08.560766 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.560723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.560996 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.560783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mffj\" (UniqueName: \"kubernetes.io/projected/36122574-48f7-4d76-a678-783a3b20c141-kube-api-access-9mffj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.560996 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.560833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.560996 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.560888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.560996 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.560967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.561151 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.561048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36122574-48f7-4d76-a678-783a3b20c141-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662240 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662240 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mffj\" (UniqueName: \"kubernetes.io/projected/36122574-48f7-4d76-a678-783a3b20c141-kube-api-access-9mffj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662240 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662240 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662587 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662587 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36122574-48f7-4d76-a678-783a3b20c141-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662587 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662558    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662740 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662711    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.662796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.662736    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.664670 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.664646    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36122574-48f7-4d76-a678-783a3b20c141-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.664966 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.664949    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36122574-48f7-4d76-a678-783a3b20c141-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.671569 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.671545    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mffj\" (UniqueName: \"kubernetes.io/projected/36122574-48f7-4d76-a678-783a3b20c141-kube-api-access-9mffj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kkdxm\" (UID: \"36122574-48f7-4d76-a678-783a3b20c141\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.702595 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.702564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:08.853725 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:08.853697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"]
Jul 07 18:06:08.855992 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:06:08.855965    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36122574_48f7_4d76_a678_783a3b20c141.slice/crio-9cacf482df52cfa0596d7b359791b40f819b357c164339e237f1a5d65f2a0a8b WatchSource:0}: Error finding container 9cacf482df52cfa0596d7b359791b40f819b357c164339e237f1a5d65f2a0a8b: Status 404 returned error can't find the container with id 9cacf482df52cfa0596d7b359791b40f819b357c164339e237f1a5d65f2a0a8b
Jul 07 18:06:09.282203 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:09.282166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm" event={"ID":"36122574-48f7-4d76-a678-783a3b20c141","Type":"ContainerStarted","Data":"668af942b2327f150583aec692437ced35fd5b8183e5f47118e89b650e3dbb92"}
Jul 07 18:06:09.282373 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:09.282221    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm" event={"ID":"36122574-48f7-4d76-a678-783a3b20c141","Type":"ContainerStarted","Data":"9cacf482df52cfa0596d7b359791b40f819b357c164339e237f1a5d65f2a0a8b"}
Jul 07 18:06:09.302262 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:09.302208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm" podStartSLOduration=1.05673171 podStartE2EDuration="1.302193589s" totalImagesPullingTime="245.461879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:06:09.300510035 +0000 UTC m=+731.190870760" watchObservedRunningTime="2026-07-07 18:06:09.302193589 +0000 UTC m=+731.192554312"
Jul 07 18:06:18.703422 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:18.703387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:18.703980 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:18.703471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:18.718476 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:18.718446    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:06:19.358920 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:06:19.358863    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kkdxm"
Jul 07 18:07:13.331977 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.331932    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84d55d9f9f-mrfp7"]
Jul 07 18:07:13.334874 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.334854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.345602 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.345574    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84d55d9f9f-mrfp7"]
Jul 07 18:07:13.364428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.364399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1d03d2a-0bf7-495e-bf44-49e1d17125d9-tls-cert\") pod \"authorino-84d55d9f9f-mrfp7\" (UID: \"a1d03d2a-0bf7-495e-bf44-49e1d17125d9\") " pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.364603 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.364456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96k9m\" (UniqueName: \"kubernetes.io/projected/a1d03d2a-0bf7-495e-bf44-49e1d17125d9-kube-api-access-96k9m\") pod \"authorino-84d55d9f9f-mrfp7\" (UID: \"a1d03d2a-0bf7-495e-bf44-49e1d17125d9\") " pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.465982 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.465935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1d03d2a-0bf7-495e-bf44-49e1d17125d9-tls-cert\") pod \"authorino-84d55d9f9f-mrfp7\" (UID: \"a1d03d2a-0bf7-495e-bf44-49e1d17125d9\") " pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.466164 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.466016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96k9m\" (UniqueName: \"kubernetes.io/projected/a1d03d2a-0bf7-495e-bf44-49e1d17125d9-kube-api-access-96k9m\") pod \"authorino-84d55d9f9f-mrfp7\" (UID: \"a1d03d2a-0bf7-495e-bf44-49e1d17125d9\") " pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.468691 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.468658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a1d03d2a-0bf7-495e-bf44-49e1d17125d9-tls-cert\") pod \"authorino-84d55d9f9f-mrfp7\" (UID: \"a1d03d2a-0bf7-495e-bf44-49e1d17125d9\") " pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.474579 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.474546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96k9m\" (UniqueName: \"kubernetes.io/projected/a1d03d2a-0bf7-495e-bf44-49e1d17125d9-kube-api-access-96k9m\") pod \"authorino-84d55d9f9f-mrfp7\" (UID: \"a1d03d2a-0bf7-495e-bf44-49e1d17125d9\") " pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.646273 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.646220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84d55d9f9f-mrfp7"
Jul 07 18:07:13.810413 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:13.810380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84d55d9f9f-mrfp7"]
Jul 07 18:07:13.810980 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:07:13.810957    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1d03d2a_0bf7_495e_bf44_49e1d17125d9.slice/crio-cf2904b87376f1b06fdd539d2c6f61411a77a950e06c22cb35aed2e5e4bd8d96 WatchSource:0}: Error finding container cf2904b87376f1b06fdd539d2c6f61411a77a950e06c22cb35aed2e5e4bd8d96: Status 404 returned error can't find the container with id cf2904b87376f1b06fdd539d2c6f61411a77a950e06c22cb35aed2e5e4bd8d96
Jul 07 18:07:14.681431 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:14.681369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84d55d9f9f-mrfp7" event={"ID":"a1d03d2a-0bf7-495e-bf44-49e1d17125d9","Type":"ContainerStarted","Data":"09bb77b77aeef1944cde9e2260b0f8b374a7c059d4dfaa2dda04bdb394244770"}
Jul 07 18:07:14.681952 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:14.681454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84d55d9f9f-mrfp7" event={"ID":"a1d03d2a-0bf7-495e-bf44-49e1d17125d9","Type":"ContainerStarted","Data":"cf2904b87376f1b06fdd539d2c6f61411a77a950e06c22cb35aed2e5e4bd8d96"}
Jul 07 18:07:14.705270 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:14.705189    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84d55d9f9f-mrfp7" podStartSLOduration=1.248950146 podStartE2EDuration="1.705164558s" totalImagesPullingTime="456.214412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:07:14.703439062 +0000 UTC m=+796.593799787" watchObservedRunningTime="2026-07-07 18:07:14.705164558 +0000 UTC m=+796.595525282"
Jul 07 18:07:14.741839 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:14.741614    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c8998946-f9nn4"]
Jul 07 18:07:14.742548 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:14.742178    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-57c8998946-f9nn4" podUID="9776a412-1bda-4fc7-80d6-359d90351c0e" containerName="authorino" containerID="cri-o://8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1" gracePeriod=30
Jul 07 18:07:15.023402 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.023371    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:07:15.084517 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.080218    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw8xz\" (UniqueName: \"kubernetes.io/projected/9776a412-1bda-4fc7-80d6-359d90351c0e-kube-api-access-xw8xz\") pod \"9776a412-1bda-4fc7-80d6-359d90351c0e\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") "
Jul 07 18:07:15.084517 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.080386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9776a412-1bda-4fc7-80d6-359d90351c0e-tls-cert\") pod \"9776a412-1bda-4fc7-80d6-359d90351c0e\" (UID: \"9776a412-1bda-4fc7-80d6-359d90351c0e\") "
Jul 07 18:07:15.085719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.085674    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9776a412-1bda-4fc7-80d6-359d90351c0e-kube-api-access-xw8xz" (OuterVolumeSpecName: "kube-api-access-xw8xz") pod "9776a412-1bda-4fc7-80d6-359d90351c0e" (UID: "9776a412-1bda-4fc7-80d6-359d90351c0e"). InnerVolumeSpecName "kube-api-access-xw8xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:07:15.096677 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.096638    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9776a412-1bda-4fc7-80d6-359d90351c0e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9776a412-1bda-4fc7-80d6-359d90351c0e" (UID: "9776a412-1bda-4fc7-80d6-359d90351c0e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:07:15.182153 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.182112    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9776a412-1bda-4fc7-80d6-359d90351c0e-tls-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:07:15.182153 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.182148    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw8xz\" (UniqueName: \"kubernetes.io/projected/9776a412-1bda-4fc7-80d6-359d90351c0e-kube-api-access-xw8xz\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:07:15.688577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.688539    2580 generic.go:356] "Generic (PLEG): container finished" podID="9776a412-1bda-4fc7-80d6-359d90351c0e" containerID="8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1" exitCode=0
Jul 07 18:07:15.689037 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.688588    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c8998946-f9nn4"
Jul 07 18:07:15.689037 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.688623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c8998946-f9nn4" event={"ID":"9776a412-1bda-4fc7-80d6-359d90351c0e","Type":"ContainerDied","Data":"8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1"}
Jul 07 18:07:15.689037 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.688673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c8998946-f9nn4" event={"ID":"9776a412-1bda-4fc7-80d6-359d90351c0e","Type":"ContainerDied","Data":"85eabad48bb3a5d36bc502a40983d08be2ae79f476213c2a88635941ba9da542"}
Jul 07 18:07:15.689037 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.688691    2580 scope.go:117] "RemoveContainer" containerID="8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1"
Jul 07 18:07:15.699487 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.699460    2580 scope.go:117] "RemoveContainer" containerID="8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1"
Jul 07 18:07:15.699829 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:07:15.699806    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1\": container with ID starting with 8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1 not found: ID does not exist" containerID="8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1"
Jul 07 18:07:15.699928 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.699840    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1"} err="failed to get container status \"8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1\": rpc error: code = NotFound desc = could not find container \"8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1\": container with ID starting with 8e70b2f8b6f55f95d7b82fee124aac9bf718f8536dde8d4b7c9538a44845a0c1 not found: ID does not exist"
Jul 07 18:07:15.714679 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.714635    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c8998946-f9nn4"]
Jul 07 18:07:15.722061 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:15.722034    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57c8998946-f9nn4"]
Jul 07 18:07:16.596875 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:07:16.596837    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9776a412-1bda-4fc7-80d6-359d90351c0e" path="/var/lib/kubelet/pods/9776a412-1bda-4fc7-80d6-359d90351c0e/volumes"
Jul 07 18:08:26.611281 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.611245    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5594d854cf-2hgvl"]
Jul 07 18:08:26.611756 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.611476    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5594d854cf-2hgvl" podUID="ed7cf592-2a20-4378-af9d-ba58422c2b8c" containerName="manager" containerID="cri-o://3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419" gracePeriod=30
Jul 07 18:08:26.880200 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.880174    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:08:26.965675 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.965643    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed7cf592-2a20-4378-af9d-ba58422c2b8c-webhook-cert\") pod \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") "
Jul 07 18:08:26.965885 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.965744    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrnbj\" (UniqueName: \"kubernetes.io/projected/ed7cf592-2a20-4378-af9d-ba58422c2b8c-kube-api-access-wrnbj\") pod \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\" (UID: \"ed7cf592-2a20-4378-af9d-ba58422c2b8c\") "
Jul 07 18:08:26.968126 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.968068    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7cf592-2a20-4378-af9d-ba58422c2b8c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ed7cf592-2a20-4378-af9d-ba58422c2b8c" (UID: "ed7cf592-2a20-4378-af9d-ba58422c2b8c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:08:26.968126 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:26.968107    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7cf592-2a20-4378-af9d-ba58422c2b8c-kube-api-access-wrnbj" (OuterVolumeSpecName: "kube-api-access-wrnbj") pod "ed7cf592-2a20-4378-af9d-ba58422c2b8c" (UID: "ed7cf592-2a20-4378-af9d-ba58422c2b8c"). InnerVolumeSpecName "kube-api-access-wrnbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:08:27.067158 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.067103    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed7cf592-2a20-4378-af9d-ba58422c2b8c-webhook-cert\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:08:27.067158 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.067147    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrnbj\" (UniqueName: \"kubernetes.io/projected/ed7cf592-2a20-4378-af9d-ba58422c2b8c-kube-api-access-wrnbj\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:08:27.121061 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.120955    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed7cf592-2a20-4378-af9d-ba58422c2b8c" containerID="3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419" exitCode=0
Jul 07 18:08:27.121061 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.120998    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5594d854cf-2hgvl"
Jul 07 18:08:27.121061 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.121027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5594d854cf-2hgvl" event={"ID":"ed7cf592-2a20-4378-af9d-ba58422c2b8c","Type":"ContainerDied","Data":"3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419"}
Jul 07 18:08:27.121368 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.121085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5594d854cf-2hgvl" event={"ID":"ed7cf592-2a20-4378-af9d-ba58422c2b8c","Type":"ContainerDied","Data":"2ad887c1f0f0415a47c78f4ed3052ff825d0384cce3b6a0b329de6c1fe48bfcd"}
Jul 07 18:08:27.121368 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.121137    2580 scope.go:117] "RemoveContainer" containerID="3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419"
Jul 07 18:08:27.130719 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.130697    2580 scope.go:117] "RemoveContainer" containerID="3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419"
Jul 07 18:08:27.131063 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:08:27.131042    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419\": container with ID starting with 3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419 not found: ID does not exist" containerID="3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419"
Jul 07 18:08:27.131138 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.131077    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419"} err="failed to get container status \"3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419\": rpc error: code = NotFound desc = could not find container \"3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419\": container with ID starting with 3ed39c9e895bdeef74b9e385d99df38dcc83d6fc9bf0b668496cbdb631a50419 not found: ID does not exist"
Jul 07 18:08:27.143659 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.143612    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5594d854cf-2hgvl"]
Jul 07 18:08:27.148321 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.148291    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5594d854cf-2hgvl"]
Jul 07 18:08:27.943476 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.943440    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5594d854cf-xt256"]
Jul 07 18:08:27.943870 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.943863    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9776a412-1bda-4fc7-80d6-359d90351c0e" containerName="authorino"
Jul 07 18:08:27.943942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.943874    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9776a412-1bda-4fc7-80d6-359d90351c0e" containerName="authorino"
Jul 07 18:08:27.943942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.943893    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7cf592-2a20-4378-af9d-ba58422c2b8c" containerName="manager"
Jul 07 18:08:27.943942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.943899    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7cf592-2a20-4378-af9d-ba58422c2b8c" containerName="manager"
Jul 07 18:08:27.944047 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.943997    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7cf592-2a20-4378-af9d-ba58422c2b8c" containerName="manager"
Jul 07 18:08:27.944047 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.944010    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9776a412-1bda-4fc7-80d6-359d90351c0e" containerName="authorino"
Jul 07 18:08:27.946698 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.946680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:27.950976 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.950952    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 18:08:27.951130 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.950951    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tm82w\""
Jul 07 18:08:27.951130 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.950952    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 18:08:27.955835 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:27.955810    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5594d854cf-xt256"]
Jul 07 18:08:28.077353 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.077311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f76511b-6c49-487d-b72d-e03e53b4ce1d-webhook-cert\") pod \"maas-controller-5594d854cf-xt256\" (UID: \"8f76511b-6c49-487d-b72d-e03e53b4ce1d\") " pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.077516 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.077400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgtcj\" (UniqueName: \"kubernetes.io/projected/8f76511b-6c49-487d-b72d-e03e53b4ce1d-kube-api-access-tgtcj\") pod \"maas-controller-5594d854cf-xt256\" (UID: \"8f76511b-6c49-487d-b72d-e03e53b4ce1d\") " pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.177921 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.177874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f76511b-6c49-487d-b72d-e03e53b4ce1d-webhook-cert\") pod \"maas-controller-5594d854cf-xt256\" (UID: \"8f76511b-6c49-487d-b72d-e03e53b4ce1d\") " pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.178135 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.177982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgtcj\" (UniqueName: \"kubernetes.io/projected/8f76511b-6c49-487d-b72d-e03e53b4ce1d-kube-api-access-tgtcj\") pod \"maas-controller-5594d854cf-xt256\" (UID: \"8f76511b-6c49-487d-b72d-e03e53b4ce1d\") " pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.180545 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.180518    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f76511b-6c49-487d-b72d-e03e53b4ce1d-webhook-cert\") pod \"maas-controller-5594d854cf-xt256\" (UID: \"8f76511b-6c49-487d-b72d-e03e53b4ce1d\") " pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.189818 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.189790    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgtcj\" (UniqueName: \"kubernetes.io/projected/8f76511b-6c49-487d-b72d-e03e53b4ce1d-kube-api-access-tgtcj\") pod \"maas-controller-5594d854cf-xt256\" (UID: \"8f76511b-6c49-487d-b72d-e03e53b4ce1d\") " pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.258525 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.258434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:28.425046 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.425013    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5594d854cf-xt256"]
Jul 07 18:08:28.426358 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:08:28.426335    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f76511b_6c49_487d_b72d_e03e53b4ce1d.slice/crio-92d5b5653507bd3ec78b3a1729c3fc55425580485ecfedfe5d7638aabbced744 WatchSource:0}: Error finding container 92d5b5653507bd3ec78b3a1729c3fc55425580485ecfedfe5d7638aabbced744: Status 404 returned error can't find the container with id 92d5b5653507bd3ec78b3a1729c3fc55425580485ecfedfe5d7638aabbced744
Jul 07 18:08:28.595995 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:28.595965    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed7cf592-2a20-4378-af9d-ba58422c2b8c" path="/var/lib/kubelet/pods/ed7cf592-2a20-4378-af9d-ba58422c2b8c/volumes"
Jul 07 18:08:29.134721 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:29.134682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5594d854cf-xt256" event={"ID":"8f76511b-6c49-487d-b72d-e03e53b4ce1d","Type":"ContainerStarted","Data":"58c849893fc736f067f69be4caa51dbfd83482971b62360324d6c59c3d0abb24"}
Jul 07 18:08:29.135176 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:29.134764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5594d854cf-xt256" event={"ID":"8f76511b-6c49-487d-b72d-e03e53b4ce1d","Type":"ContainerStarted","Data":"92d5b5653507bd3ec78b3a1729c3fc55425580485ecfedfe5d7638aabbced744"}
Jul 07 18:08:29.135176 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:29.134834    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:08:29.162365 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:29.162303    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5594d854cf-xt256" podStartSLOduration=1.632710292 podStartE2EDuration="2.162285255s" totalImagesPullingTime="529.574963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:08:29.160525003 +0000 UTC m=+871.050885729" watchObservedRunningTime="2026-07-07 18:08:29.162285255 +0000 UTC m=+871.052645979"
Jul 07 18:08:40.146783 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:08:40.146747    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5594d854cf-xt256"
Jul 07 18:15:00.161328 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.161243    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724135-d579w"]
Jul 07 18:15:00.164763 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.164746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:00.175529 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.175501    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724135-d579w"]
Jul 07 18:15:00.262791 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.262743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfvmr\" (UniqueName: \"kubernetes.io/projected/72011f40-2da4-419d-a639-08c3c3b4e163-kube-api-access-dfvmr\") pod \"maas-api-key-cleanup-29724135-d579w\" (UID: \"72011f40-2da4-419d-a639-08c3c3b4e163\") " pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:00.363449 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.363419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfvmr\" (UniqueName: \"kubernetes.io/projected/72011f40-2da4-419d-a639-08c3c3b4e163-kube-api-access-dfvmr\") pod \"maas-api-key-cleanup-29724135-d579w\" (UID: \"72011f40-2da4-419d-a639-08c3c3b4e163\") " pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:00.375337 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.375301    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfvmr\" (UniqueName: \"kubernetes.io/projected/72011f40-2da4-419d-a639-08c3c3b4e163-kube-api-access-dfvmr\") pod \"maas-api-key-cleanup-29724135-d579w\" (UID: \"72011f40-2da4-419d-a639-08c3c3b4e163\") " pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:00.475408 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.475318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:00.627185 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.627157    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724135-d579w"]
Jul 07 18:15:00.629551 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:15:00.629519    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72011f40_2da4_419d_a639_08c3c3b4e163.slice/crio-c2143564b624d8b0721b38528a18ae2dc086bebf8659678a56ab90157af66b5b WatchSource:0}: Error finding container c2143564b624d8b0721b38528a18ae2dc086bebf8659678a56ab90157af66b5b: Status 404 returned error can't find the container with id c2143564b624d8b0721b38528a18ae2dc086bebf8659678a56ab90157af66b5b
Jul 07 18:15:00.631818 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:00.631797    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 18:15:01.472786 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:01.472734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724135-d579w" event={"ID":"72011f40-2da4-419d-a639-08c3c3b4e163","Type":"ContainerStarted","Data":"c2143564b624d8b0721b38528a18ae2dc086bebf8659678a56ab90157af66b5b"}
Jul 07 18:15:03.486787 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:03.486747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724135-d579w" event={"ID":"72011f40-2da4-419d-a639-08c3c3b4e163","Type":"ContainerStarted","Data":"60adb948fee6d7ad261b244ece98b025dc6b259a3e3c0c5f8033a5dbf14af891"}
Jul 07 18:15:03.507555 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:03.507493    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724135-d579w" podStartSLOduration=1.362801467 podStartE2EDuration="3.507473629s" totalImagesPullingTime="2.144672162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:15:03.506690424 +0000 UTC m=+1265.397051148" watchObservedRunningTime="2026-07-07 18:15:03.507473629 +0000 UTC m=+1265.397834353"
Jul 07 18:15:04.494650 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:04.494611    2580 generic.go:356] "Generic (PLEG): container finished" podID="72011f40-2da4-419d-a639-08c3c3b4e163" containerID="60adb948fee6d7ad261b244ece98b025dc6b259a3e3c0c5f8033a5dbf14af891" exitCode=0
Jul 07 18:15:04.495147 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:04.494677    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724135-d579w" event={"ID":"72011f40-2da4-419d-a639-08c3c3b4e163","Type":"ContainerDied","Data":"60adb948fee6d7ad261b244ece98b025dc6b259a3e3c0c5f8033a5dbf14af891"}
Jul 07 18:15:05.662737 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:05.662699    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:05.716496 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:05.716448    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfvmr\" (UniqueName: \"kubernetes.io/projected/72011f40-2da4-419d-a639-08c3c3b4e163-kube-api-access-dfvmr\") pod \"72011f40-2da4-419d-a639-08c3c3b4e163\" (UID: \"72011f40-2da4-419d-a639-08c3c3b4e163\") "
Jul 07 18:15:05.718740 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:05.718699    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72011f40-2da4-419d-a639-08c3c3b4e163-kube-api-access-dfvmr" (OuterVolumeSpecName: "kube-api-access-dfvmr") pod "72011f40-2da4-419d-a639-08c3c3b4e163" (UID: "72011f40-2da4-419d-a639-08c3c3b4e163"). InnerVolumeSpecName "kube-api-access-dfvmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:15:05.817827 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:05.817728    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfvmr\" (UniqueName: \"kubernetes.io/projected/72011f40-2da4-419d-a639-08c3c3b4e163-kube-api-access-dfvmr\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:15:06.509437 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:06.509396    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724135-d579w"
Jul 07 18:15:06.509639 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:06.509404    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724135-d579w" event={"ID":"72011f40-2da4-419d-a639-08c3c3b4e163","Type":"ContainerDied","Data":"c2143564b624d8b0721b38528a18ae2dc086bebf8659678a56ab90157af66b5b"}
Jul 07 18:15:06.509639 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:15:06.509567    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2143564b624d8b0721b38528a18ae2dc086bebf8659678a56ab90157af66b5b"
Jul 07 18:24:10.693174 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.693134    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"]
Jul 07 18:24:10.693712 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.693692    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72011f40-2da4-419d-a639-08c3c3b4e163" containerName="cleanup"
Jul 07 18:24:10.693712 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.693713    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="72011f40-2da4-419d-a639-08c3c3b4e163" containerName="cleanup"
Jul 07 18:24:10.693796 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.693783    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="72011f40-2da4-419d-a639-08c3c3b4e163" containerName="cleanup"
Jul 07 18:24:10.696998 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.696981    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.701866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.701842    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d1f3c1c2\"/\"openshift-service-ca.crt\""
Jul 07 18:24:10.702022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.701863    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d1f3c1c2\"/\"e2e-lifecycle-model-d1f3c1c2-kserve-self-signed-certs\""
Jul 07 18:24:10.702022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.701842    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d1f3c1c2\"/\"default-dockercfg-brhrj\""
Jul 07 18:24:10.702022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.701887    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d1f3c1c2\"/\"kube-root-ca.crt\""
Jul 07 18:24:10.717456 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.717427    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"]
Jul 07 18:24:10.754141 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.754106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tls-certs\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.754351 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.754169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tmp-dir\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.754351 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.754210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-home\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.754351 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.754237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-model-cache\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.754351 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.754308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-dshm\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.754552 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.754375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76twj\" (UniqueName: \"kubernetes.io/projected/8c699eb1-92c7-444b-aa91-e5529ba32fd2-kube-api-access-76twj\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855409 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-model-cache\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855626 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-dshm\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855626 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855497    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76twj\" (UniqueName: \"kubernetes.io/projected/8c699eb1-92c7-444b-aa91-e5529ba32fd2-kube-api-access-76twj\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855626 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tls-certs\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855626 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tmp-dir\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855859 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-home\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.855859 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855835    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-model-cache\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.856005 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.855974    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tmp-dir\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.856080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.856058    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-home\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.857878 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.857853    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-dshm\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.858334 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.858314    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tls-certs\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:10.865317 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:10.865288    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76twj\" (UniqueName: \"kubernetes.io/projected/8c699eb1-92c7-444b-aa91-e5529ba32fd2-kube-api-access-76twj\") pod \"e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") " pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:11.007087 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:11.006987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:11.156163 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:11.156129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"]
Jul 07 18:24:11.157573 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:24:11.157552    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c699eb1_92c7_444b_aa91_e5529ba32fd2.slice/crio-3cacf2b26433078d6e96b861a121f79d8498203cb0d3aaac872aca2e538459e3 WatchSource:0}: Error finding container 3cacf2b26433078d6e96b861a121f79d8498203cb0d3aaac872aca2e538459e3: Status 404 returned error can't find the container with id 3cacf2b26433078d6e96b861a121f79d8498203cb0d3aaac872aca2e538459e3
Jul 07 18:24:11.689399 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:11.689361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5" event={"ID":"8c699eb1-92c7-444b-aa91-e5529ba32fd2","Type":"ContainerStarted","Data":"cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88"}
Jul 07 18:24:11.689399 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:11.689405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5" event={"ID":"8c699eb1-92c7-444b-aa91-e5529ba32fd2","Type":"ContainerStarted","Data":"3cacf2b26433078d6e96b861a121f79d8498203cb0d3aaac872aca2e538459e3"}
Jul 07 18:24:11.711075 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:11.711011    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5" podStartSLOduration=1.7109948560000001 podStartE2EDuration="1.710994856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:24:11.708639857 +0000 UTC m=+1813.599000582" watchObservedRunningTime="2026-07-07 18:24:11.710994856 +0000 UTC m=+1813.601355580"
Jul 07 18:24:21.007559 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:21.007511    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:21.008046 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:21.007800    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:21.020747 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:21.020716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:21.763290 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:21.763260    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:41.302760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:41.302724    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"]
Jul 07 18:24:56.324340 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.324290    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5" podUID="8c699eb1-92c7-444b-aa91-e5529ba32fd2" containerName="main" containerID="cri-o://cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88" gracePeriod=45
Jul 07 18:24:56.585632 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.585564    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:56.590017 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.589986    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-dshm\") pod \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") "
Jul 07 18:24:56.590148 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590038    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-model-cache\") pod \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") "
Jul 07 18:24:56.590148 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590077    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tls-certs\") pod \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") "
Jul 07 18:24:56.590269 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590166    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-home\") pod \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") "
Jul 07 18:24:56.590269 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76twj\" (UniqueName: \"kubernetes.io/projected/8c699eb1-92c7-444b-aa91-e5529ba32fd2-kube-api-access-76twj\") pod \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") "
Jul 07 18:24:56.590269 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590266    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tmp-dir\") pod \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\" (UID: \"8c699eb1-92c7-444b-aa91-e5529ba32fd2\") "
Jul 07 18:24:56.590456 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590360    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c699eb1-92c7-444b-aa91-e5529ba32fd2" (UID: "8c699eb1-92c7-444b-aa91-e5529ba32fd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:24:56.590599 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590577    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-model-cache\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:24:56.590599 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590582    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-home" (OuterVolumeSpecName: "home") pod "8c699eb1-92c7-444b-aa91-e5529ba32fd2" (UID: "8c699eb1-92c7-444b-aa91-e5529ba32fd2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:24:56.590733 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.590706    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c699eb1-92c7-444b-aa91-e5529ba32fd2" (UID: "8c699eb1-92c7-444b-aa91-e5529ba32fd2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:24:56.592607 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.592584    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-dshm" (OuterVolumeSpecName: "dshm") pod "8c699eb1-92c7-444b-aa91-e5529ba32fd2" (UID: "8c699eb1-92c7-444b-aa91-e5529ba32fd2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:24:56.592710 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.592636    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c699eb1-92c7-444b-aa91-e5529ba32fd2" (UID: "8c699eb1-92c7-444b-aa91-e5529ba32fd2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:24:56.593186 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.593169    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c699eb1-92c7-444b-aa91-e5529ba32fd2-kube-api-access-76twj" (OuterVolumeSpecName: "kube-api-access-76twj") pod "8c699eb1-92c7-444b-aa91-e5529ba32fd2" (UID: "8c699eb1-92c7-444b-aa91-e5529ba32fd2"). InnerVolumeSpecName "kube-api-access-76twj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:24:56.691527 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.691489    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-home\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:24:56.691527 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.691522    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76twj\" (UniqueName: \"kubernetes.io/projected/8c699eb1-92c7-444b-aa91-e5529ba32fd2-kube-api-access-76twj\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:24:56.691527 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.691534    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tmp-dir\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:24:56.691783 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.691544    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c699eb1-92c7-444b-aa91-e5529ba32fd2-dshm\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:24:56.691783 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.691553    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c699eb1-92c7-444b-aa91-e5529ba32fd2-tls-certs\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:24:56.961122 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.961089    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c699eb1-92c7-444b-aa91-e5529ba32fd2" containerID="cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88" exitCode=0
Jul 07 18:24:56.961315 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.961134    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"
Jul 07 18:24:56.961315 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.961184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5" event={"ID":"8c699eb1-92c7-444b-aa91-e5529ba32fd2","Type":"ContainerDied","Data":"cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88"}
Jul 07 18:24:56.961315 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.961231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5" event={"ID":"8c699eb1-92c7-444b-aa91-e5529ba32fd2","Type":"ContainerDied","Data":"3cacf2b26433078d6e96b861a121f79d8498203cb0d3aaac872aca2e538459e3"}
Jul 07 18:24:56.961315 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.961252    2580 scope.go:117] "RemoveContainer" containerID="cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88"
Jul 07 18:24:56.971680 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.971659    2580 scope.go:117] "RemoveContainer" containerID="cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88"
Jul 07 18:24:56.972063 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:24:56.972039    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88\": container with ID starting with cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88 not found: ID does not exist" containerID="cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88"
Jul 07 18:24:56.972156 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.972076    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88"} err="failed to get container status \"cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88\": rpc error: code = NotFound desc = could not find container \"cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88\": container with ID starting with cba69f951d9f1b7a07b1c72a982033920b832fa3a23aea54aab999991ec67f88 not found: ID does not exist"
Jul 07 18:24:56.984505 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.984478    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"]
Jul 07 18:24:56.996562 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:56.996528    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d1f3c1c2/e2e-lifecycle-model-d1f3c1c2-kserve-7f8f78dd4c-mjzf5"]
Jul 07 18:24:58.602577 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:24:58.602539    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c699eb1-92c7-444b-aa91-e5529ba32fd2" path="/var/lib/kubelet/pods/8c699eb1-92c7-444b-aa91-e5529ba32fd2/volumes"
Jul 07 18:26:54.677425 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.677343    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"]
Jul 07 18:26:54.678082 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.678061    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c699eb1-92c7-444b-aa91-e5529ba32fd2" containerName="main"
Jul 07 18:26:54.678138 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.678087    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c699eb1-92c7-444b-aa91-e5529ba32fd2" containerName="main"
Jul 07 18:26:54.678205 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.678193    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c699eb1-92c7-444b-aa91-e5529ba32fd2" containerName="main"
Jul 07 18:26:54.681831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.681805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.684379 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.684352    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-95ab96\"/\"openshift-service-ca.crt\""
Jul 07 18:26:54.684379 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.684366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-95ab96\"/\"default-dockercfg-9jpx9\""
Jul 07 18:26:54.685935 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.685885    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-95ab96\"/\"kube-root-ca.crt\""
Jul 07 18:26:54.686695 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.686324    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-95ab96\"/\"test-model-95ab96-kserve-self-signed-certs\""
Jul 07 18:26:54.694566 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.693610    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"]
Jul 07 18:26:54.805141 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.805098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tls-certs\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.805141 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.805144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-home\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.805419 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.805171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-model-cache\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.805419 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.805270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tmp-dir\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.805419 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.805326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-dshm\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.805570 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.805433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wrt5\" (UniqueName: \"kubernetes.io/projected/1940c4d7-e0a7-4a0f-a130-e8718be99beb-kube-api-access-5wrt5\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906131 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tmp-dir\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-dshm\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wrt5\" (UniqueName: \"kubernetes.io/projected/1940c4d7-e0a7-4a0f-a130-e8718be99beb-kube-api-access-5wrt5\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tls-certs\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-home\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906348 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-model-cache\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906625 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906595    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tmp-dir\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906682 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906664    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-home\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.906725 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.906700    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-model-cache\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.908665 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.908642    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-dshm\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.908998 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.908967    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tls-certs\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.915296 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.915265    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wrt5\" (UniqueName: \"kubernetes.io/projected/1940c4d7-e0a7-4a0f-a130-e8718be99beb-kube-api-access-5wrt5\") pod \"test-model-95ab96-kserve-799f5d6965-656xc\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") " pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:54.997483 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:54.997384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:26:55.147609 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:55.147575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"]
Jul 07 18:26:55.149091 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:26:55.149059    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1940c4d7_e0a7_4a0f_a130_e8718be99beb.slice/crio-7b14788a2bba2364393d93e5c1e67afb0da22ee2ec421d8d15790032f84ca304 WatchSource:0}: Error finding container 7b14788a2bba2364393d93e5c1e67afb0da22ee2ec421d8d15790032f84ca304: Status 404 returned error can't find the container with id 7b14788a2bba2364393d93e5c1e67afb0da22ee2ec421d8d15790032f84ca304
Jul 07 18:26:55.673176 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:55.673139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc" event={"ID":"1940c4d7-e0a7-4a0f-a130-e8718be99beb","Type":"ContainerStarted","Data":"71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e"}
Jul 07 18:26:55.673176 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:55.673185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc" event={"ID":"1940c4d7-e0a7-4a0f-a130-e8718be99beb","Type":"ContainerStarted","Data":"7b14788a2bba2364393d93e5c1e67afb0da22ee2ec421d8d15790032f84ca304"}
Jul 07 18:26:55.699717 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:26:55.699640    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc" podStartSLOduration=1.69962295 podStartE2EDuration="1.69962295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:26:55.697410731 +0000 UTC m=+1977.587771455" watchObservedRunningTime="2026-07-07 18:26:55.69962295 +0000 UTC m=+1977.589983674"
Jul 07 18:27:04.997729 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:27:04.997693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:27:04.998175 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:27:04.997749    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:27:05.012311 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:27:05.012278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:27:05.744758 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:27:05.744724    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:28:21.395471 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:21.395375    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"]
Jul 07 18:28:36.419101 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.419033    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc" podUID="1940c4d7-e0a7-4a0f-a130-e8718be99beb" containerName="main" containerID="cri-o://71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e" gracePeriod=45
Jul 07 18:28:36.677883 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.677800    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:28:36.789510 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-dshm\") pod \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") "
Jul 07 18:28:36.789510 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wrt5\" (UniqueName: \"kubernetes.io/projected/1940c4d7-e0a7-4a0f-a130-e8718be99beb-kube-api-access-5wrt5\") pod \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") "
Jul 07 18:28:36.789733 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789570    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-home\") pod \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") "
Jul 07 18:28:36.789733 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tmp-dir\") pod \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") "
Jul 07 18:28:36.789825 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789790    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-model-cache\") pod \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") "
Jul 07 18:28:36.789825 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789798    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-home" (OuterVolumeSpecName: "home") pod "1940c4d7-e0a7-4a0f-a130-e8718be99beb" (UID: "1940c4d7-e0a7-4a0f-a130-e8718be99beb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:28:36.789926 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tls-certs\") pod \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\" (UID: \"1940c4d7-e0a7-4a0f-a130-e8718be99beb\") "
Jul 07 18:28:36.790019 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.789965    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1940c4d7-e0a7-4a0f-a130-e8718be99beb" (UID: "1940c4d7-e0a7-4a0f-a130-e8718be99beb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:28:36.790064 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.790034    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-model-cache" (OuterVolumeSpecName: "model-cache") pod "1940c4d7-e0a7-4a0f-a130-e8718be99beb" (UID: "1940c4d7-e0a7-4a0f-a130-e8718be99beb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:28:36.790257 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.790240    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-model-cache\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:28:36.790304 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.790261    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-home\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:28:36.790304 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.790270    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tmp-dir\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:28:36.792218 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.792185    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1940c4d7-e0a7-4a0f-a130-e8718be99beb" (UID: "1940c4d7-e0a7-4a0f-a130-e8718be99beb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:28:36.792329 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.792269    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-dshm" (OuterVolumeSpecName: "dshm") pod "1940c4d7-e0a7-4a0f-a130-e8718be99beb" (UID: "1940c4d7-e0a7-4a0f-a130-e8718be99beb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:28:36.792329 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.792296    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1940c4d7-e0a7-4a0f-a130-e8718be99beb-kube-api-access-5wrt5" (OuterVolumeSpecName: "kube-api-access-5wrt5") pod "1940c4d7-e0a7-4a0f-a130-e8718be99beb" (UID: "1940c4d7-e0a7-4a0f-a130-e8718be99beb"). InnerVolumeSpecName "kube-api-access-5wrt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:28:36.891797 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.891739    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1940c4d7-e0a7-4a0f-a130-e8718be99beb-tls-certs\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:28:36.891797 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.891785    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1940c4d7-e0a7-4a0f-a130-e8718be99beb-dshm\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:28:36.891797 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:36.891795    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wrt5\" (UniqueName: \"kubernetes.io/projected/1940c4d7-e0a7-4a0f-a130-e8718be99beb-kube-api-access-5wrt5\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:28:37.278589 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.278552    2580 generic.go:356] "Generic (PLEG): container finished" podID="1940c4d7-e0a7-4a0f-a130-e8718be99beb" containerID="71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e" exitCode=0
Jul 07 18:28:37.278804 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.278603    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"
Jul 07 18:28:37.278804 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.278637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc" event={"ID":"1940c4d7-e0a7-4a0f-a130-e8718be99beb","Type":"ContainerDied","Data":"71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e"}
Jul 07 18:28:37.278804 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.278681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc" event={"ID":"1940c4d7-e0a7-4a0f-a130-e8718be99beb","Type":"ContainerDied","Data":"7b14788a2bba2364393d93e5c1e67afb0da22ee2ec421d8d15790032f84ca304"}
Jul 07 18:28:37.278804 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.278700    2580 scope.go:117] "RemoveContainer" containerID="71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e"
Jul 07 18:28:37.291878 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.291857    2580 scope.go:117] "RemoveContainer" containerID="71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e"
Jul 07 18:28:37.292191 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:28:37.292169    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e\": container with ID starting with 71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e not found: ID does not exist" containerID="71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e"
Jul 07 18:28:37.292261 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.292202    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e"} err="failed to get container status \"71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e\": rpc error: code = NotFound desc = could not find container \"71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e\": container with ID starting with 71393c858aa7b01ffb9c41134a12927bf7dc75a72924977b20e95e072a9b8a7e not found: ID does not exist"
Jul 07 18:28:37.308080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.308048    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"]
Jul 07 18:28:37.310708 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:37.310674    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-95ab96/test-model-95ab96-kserve-799f5d6965-656xc"]
Jul 07 18:28:38.596101 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:28:38.596062    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1940c4d7-e0a7-4a0f-a130-e8718be99beb" path="/var/lib/kubelet/pods/1940c4d7-e0a7-4a0f-a130-e8718be99beb/volumes"
Jul 07 18:30:00.134896 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.134809    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724150-bpcq6"]
Jul 07 18:30:00.135491 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.135420    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1940c4d7-e0a7-4a0f-a130-e8718be99beb" containerName="main"
Jul 07 18:30:00.135491 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.135440    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1940c4d7-e0a7-4a0f-a130-e8718be99beb" containerName="main"
Jul 07 18:30:00.135671 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.135522    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1940c4d7-e0a7-4a0f-a130-e8718be99beb" containerName="main"
Jul 07 18:30:00.139035 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.139009    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"]
Jul 07 18:30:00.139178 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.139132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:00.143134 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.143115    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:00.150828 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.150780    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724150-bpcq6"]
Jul 07 18:30:00.154402 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.154369    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"]
Jul 07 18:30:00.219515 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.219478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb4x9\" (UniqueName: \"kubernetes.io/projected/4435bcc1-5cec-4843-80b4-0693cfd296f9-kube-api-access-bb4x9\") pod \"maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt\" (UID: \"4435bcc1-5cec-4843-80b4-0693cfd296f9\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:00.219703 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.219535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wjb\" (UniqueName: \"kubernetes.io/projected/a93c14c1-f18f-4cae-86f8-78f2382acb20-kube-api-access-b6wjb\") pod \"maas-api-key-cleanup-29724150-bpcq6\" (UID: \"a93c14c1-f18f-4cae-86f8-78f2382acb20\") " pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:00.320765 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.320724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb4x9\" (UniqueName: \"kubernetes.io/projected/4435bcc1-5cec-4843-80b4-0693cfd296f9-kube-api-access-bb4x9\") pod \"maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt\" (UID: \"4435bcc1-5cec-4843-80b4-0693cfd296f9\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:00.321001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.320785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wjb\" (UniqueName: \"kubernetes.io/projected/a93c14c1-f18f-4cae-86f8-78f2382acb20-kube-api-access-b6wjb\") pod \"maas-api-key-cleanup-29724150-bpcq6\" (UID: \"a93c14c1-f18f-4cae-86f8-78f2382acb20\") " pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:00.331782 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.331751    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wjb\" (UniqueName: \"kubernetes.io/projected/a93c14c1-f18f-4cae-86f8-78f2382acb20-kube-api-access-b6wjb\") pod \"maas-api-key-cleanup-29724150-bpcq6\" (UID: \"a93c14c1-f18f-4cae-86f8-78f2382acb20\") " pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:00.331968 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.331884    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb4x9\" (UniqueName: \"kubernetes.io/projected/4435bcc1-5cec-4843-80b4-0693cfd296f9-kube-api-access-bb4x9\") pod \"maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt\" (UID: \"4435bcc1-5cec-4843-80b4-0693cfd296f9\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:00.453172 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.453086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:00.463710 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.463672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:00.619600 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.619560    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724150-bpcq6"]
Jul 07 18:30:00.623211 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:30:00.623167    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93c14c1_f18f_4cae_86f8_78f2382acb20.slice/crio-371a1c752309f932bc8641a14078a19e0806122d825add6c275be66e19943502 WatchSource:0}: Error finding container 371a1c752309f932bc8641a14078a19e0806122d825add6c275be66e19943502: Status 404 returned error can't find the container with id 371a1c752309f932bc8641a14078a19e0806122d825add6c275be66e19943502
Jul 07 18:30:00.650260 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.650233    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"]
Jul 07 18:30:00.661231 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:30:00.661203    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4435bcc1_5cec_4843_80b4_0693cfd296f9.slice/crio-f3381d7e48f9a4cf5149f45a79e2afdca60c9aec2b38462aa4f47021dc41cbfa WatchSource:0}: Error finding container f3381d7e48f9a4cf5149f45a79e2afdca60c9aec2b38462aa4f47021dc41cbfa: Status 404 returned error can't find the container with id f3381d7e48f9a4cf5149f45a79e2afdca60c9aec2b38462aa4f47021dc41cbfa
Jul 07 18:30:00.763478 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.763382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6" event={"ID":"a93c14c1-f18f-4cae-86f8-78f2382acb20","Type":"ContainerStarted","Data":"371a1c752309f932bc8641a14078a19e0806122d825add6c275be66e19943502"}
Jul 07 18:30:00.764580 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:00.764555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" event={"ID":"4435bcc1-5cec-4843-80b4-0693cfd296f9","Type":"ContainerStarted","Data":"f3381d7e48f9a4cf5149f45a79e2afdca60c9aec2b38462aa4f47021dc41cbfa"}
Jul 07 18:30:01.771545 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:01.771506    2580 generic.go:356] "Generic (PLEG): container finished" podID="a93c14c1-f18f-4cae-86f8-78f2382acb20" containerID="35f2dd2bc92e6293be92b02ed9d8aaf1dec61e800794f27459301d66ae3addd2" exitCode=0
Jul 07 18:30:01.771985 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:01.771592    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6" event={"ID":"a93c14c1-f18f-4cae-86f8-78f2382acb20","Type":"ContainerDied","Data":"35f2dd2bc92e6293be92b02ed9d8aaf1dec61e800794f27459301d66ae3addd2"}
Jul 07 18:30:01.773315 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:01.773288    2580 generic.go:356] "Generic (PLEG): container finished" podID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerID="4d2a421d55b70090ace86fbae397e61dad6a4d3f77e58c0c825623be0b778fed" exitCode=7
Jul 07 18:30:01.773457 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:01.773377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" event={"ID":"4435bcc1-5cec-4843-80b4-0693cfd296f9","Type":"ContainerDied","Data":"4d2a421d55b70090ace86fbae397e61dad6a4d3f77e58c0c825623be0b778fed"}
Jul 07 18:30:01.773565 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:01.773552    2580 scope.go:117] "RemoveContainer" containerID="4d2a421d55b70090ace86fbae397e61dad6a4d3f77e58c0c825623be0b778fed"
Jul 07 18:30:02.781800 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:02.781704    2580 generic.go:356] "Generic (PLEG): container finished" podID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerID="59f3435ad58f1912ec3e0bdd4bbc371224a53ea4b1c0a8f4162fd83c0d5f4d19" exitCode=7
Jul 07 18:30:02.782260 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:02.781789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" event={"ID":"4435bcc1-5cec-4843-80b4-0693cfd296f9","Type":"ContainerDied","Data":"59f3435ad58f1912ec3e0bdd4bbc371224a53ea4b1c0a8f4162fd83c0d5f4d19"}
Jul 07 18:30:02.782260 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:02.781849    2580 scope.go:117] "RemoveContainer" containerID="4d2a421d55b70090ace86fbae397e61dad6a4d3f77e58c0c825623be0b778fed"
Jul 07 18:30:02.782260 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:02.782038    2580 scope.go:117] "RemoveContainer" containerID="59f3435ad58f1912ec3e0bdd4bbc371224a53ea4b1c0a8f4162fd83c0d5f4d19"
Jul 07 18:30:02.782702 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:30:02.782298    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt_opendatahub(4435bcc1-5cec-4843-80b4-0693cfd296f9)\"" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9"
Jul 07 18:30:02.931091 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:02.931061    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:03.049107 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.049010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6wjb\" (UniqueName: \"kubernetes.io/projected/a93c14c1-f18f-4cae-86f8-78f2382acb20-kube-api-access-b6wjb\") pod \"a93c14c1-f18f-4cae-86f8-78f2382acb20\" (UID: \"a93c14c1-f18f-4cae-86f8-78f2382acb20\") "
Jul 07 18:30:03.051743 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.051705    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93c14c1-f18f-4cae-86f8-78f2382acb20-kube-api-access-b6wjb" (OuterVolumeSpecName: "kube-api-access-b6wjb") pod "a93c14c1-f18f-4cae-86f8-78f2382acb20" (UID: "a93c14c1-f18f-4cae-86f8-78f2382acb20"). InnerVolumeSpecName "kube-api-access-b6wjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:30:03.150416 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.150375    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6wjb\" (UniqueName: \"kubernetes.io/projected/a93c14c1-f18f-4cae-86f8-78f2382acb20-kube-api-access-b6wjb\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:30:03.790149 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.790118    2580 scope.go:117] "RemoveContainer" containerID="59f3435ad58f1912ec3e0bdd4bbc371224a53ea4b1c0a8f4162fd83c0d5f4d19"
Jul 07 18:30:03.790623 ip-10-0-134-243 kubenswrapper[2580]: E0707 18:30:03.790342    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt_opendatahub(4435bcc1-5cec-4843-80b4-0693cfd296f9)\"" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9"
Jul 07 18:30:03.791261 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.791241    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6"
Jul 07 18:30:03.791338 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.791253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724150-bpcq6" event={"ID":"a93c14c1-f18f-4cae-86f8-78f2382acb20","Type":"ContainerDied","Data":"371a1c752309f932bc8641a14078a19e0806122d825add6c275be66e19943502"}
Jul 07 18:30:03.791338 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:03.791299    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="371a1c752309f932bc8641a14078a19e0806122d825add6c275be66e19943502"
Jul 07 18:30:15.589504 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:15.589471    2580 scope.go:117] "RemoveContainer" containerID="59f3435ad58f1912ec3e0bdd4bbc371224a53ea4b1c0a8f4162fd83c0d5f4d19"
Jul 07 18:30:16.882104 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:16.882017    2580 generic.go:356] "Generic (PLEG): container finished" podID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerID="e8b426ed0bf6900c7392aa62191254717355a8a2ddf9d86a918b61802a4ea242" exitCode=0
Jul 07 18:30:16.882104 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:16.882090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" event={"ID":"4435bcc1-5cec-4843-80b4-0693cfd296f9","Type":"ContainerDied","Data":"e8b426ed0bf6900c7392aa62191254717355a8a2ddf9d86a918b61802a4ea242"}
Jul 07 18:30:16.882544 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:16.882158    2580 scope.go:117] "RemoveContainer" containerID="59f3435ad58f1912ec3e0bdd4bbc371224a53ea4b1c0a8f4162fd83c0d5f4d19"
Jul 07 18:30:17.916176 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:17.916141    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"]
Jul 07 18:30:18.046046 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.046019    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:18.201402 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.201286    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb4x9\" (UniqueName: \"kubernetes.io/projected/4435bcc1-5cec-4843-80b4-0693cfd296f9-kube-api-access-bb4x9\") pod \"4435bcc1-5cec-4843-80b4-0693cfd296f9\" (UID: \"4435bcc1-5cec-4843-80b4-0693cfd296f9\") "
Jul 07 18:30:18.203579 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.203556    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4435bcc1-5cec-4843-80b4-0693cfd296f9-kube-api-access-bb4x9" (OuterVolumeSpecName: "kube-api-access-bb4x9") pod "4435bcc1-5cec-4843-80b4-0693cfd296f9" (UID: "4435bcc1-5cec-4843-80b4-0693cfd296f9"). InnerVolumeSpecName "kube-api-access-bb4x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:30:18.302303 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.302260    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb4x9\" (UniqueName: \"kubernetes.io/projected/4435bcc1-5cec-4843-80b4-0693cfd296f9-kube-api-access-bb4x9\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:30:18.897536 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.897443    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt" event={"ID":"4435bcc1-5cec-4843-80b4-0693cfd296f9","Type":"ContainerDied","Data":"f3381d7e48f9a4cf5149f45a79e2afdca60c9aec2b38462aa4f47021dc41cbfa"}
Jul 07 18:30:18.897536 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.897477    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"
Jul 07 18:30:18.897536 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.897495    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3381d7e48f9a4cf5149f45a79e2afdca60c9aec2b38462aa4f47021dc41cbfa"
Jul 07 18:30:18.929468 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.929431    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"]
Jul 07 18:30:18.934438 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:18.934410    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-2b5160-29724150-4d2dt"]
Jul 07 18:30:20.595657 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:20.595616    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" path="/var/lib/kubelet/pods/4435bcc1-5cec-4843-80b4-0693cfd296f9/volumes"
Jul 07 18:30:29.678394 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678362    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"]
Jul 07 18:30:29.678788 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678781    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:30:29.678831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678792    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:30:29.678831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678808    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:30:29.678831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678814    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:30:29.679001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678832    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93c14c1-f18f-4cae-86f8-78f2382acb20" containerName="cleanup"
Jul 07 18:30:29.679001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678838    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93c14c1-f18f-4cae-86f8-78f2382acb20" containerName="cleanup"
Jul 07 18:30:29.679001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678948    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93c14c1-f18f-4cae-86f8-78f2382acb20" containerName="cleanup"
Jul 07 18:30:29.679001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678957    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:30:29.679001 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.678965    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:30:29.682319 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.682301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.686625 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.686594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-200b05\"/\"openshift-service-ca.crt\""
Jul 07 18:30:29.686790 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.686629    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-200b05\"/\"auth-test-model-200b05-kserve-self-signed-certs\""
Jul 07 18:30:29.686790 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.686716    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-200b05\"/\"kube-root-ca.crt\""
Jul 07 18:30:29.686790 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.686728    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-200b05\"/\"default-dockercfg-5hzlc\""
Jul 07 18:30:29.693704 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.693677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"]
Jul 07 18:30:29.695200 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.695179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxnsp\" (UniqueName: \"kubernetes.io/projected/8990248b-308e-48c4-ba3c-b956abf4ce32-kube-api-access-vxnsp\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.695325 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.695231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-model-cache\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.695325 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.695264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-home\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.695325 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.695317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-tmp-dir\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.695461 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.695443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-dshm\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.695501 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.695468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8990248b-308e-48c4-ba3c-b956abf4ce32-tls-certs\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796020 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.795976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-model-cache\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796020 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-home\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796262 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-tmp-dir\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796262 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-dshm\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796262 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8990248b-308e-48c4-ba3c-b956abf4ce32-tls-certs\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796262 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxnsp\" (UniqueName: \"kubernetes.io/projected/8990248b-308e-48c4-ba3c-b956abf4ce32-kube-api-access-vxnsp\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796502 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796477    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-model-cache\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-home\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.796550 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.796527    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-tmp-dir\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.798692 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.798664    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-dshm\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.798968 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.798950    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8990248b-308e-48c4-ba3c-b956abf4ce32-tls-certs\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.804755 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.804727    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxnsp\" (UniqueName: \"kubernetes.io/projected/8990248b-308e-48c4-ba3c-b956abf4ce32-kube-api-access-vxnsp\") pod \"auth-test-model-200b05-kserve-68958f45dc-wpxp9\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") " pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:29.993074 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:29.992948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:30.156528 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:30.156499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"]
Jul 07 18:30:30.158467 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:30:30.158427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8990248b_308e_48c4_ba3c_b956abf4ce32.slice/crio-64efb5fbfa1849f63f29d16f0c0102cfb6563f216d1dec93f8014ab7791d9cca WatchSource:0}: Error finding container 64efb5fbfa1849f63f29d16f0c0102cfb6563f216d1dec93f8014ab7791d9cca: Status 404 returned error can't find the container with id 64efb5fbfa1849f63f29d16f0c0102cfb6563f216d1dec93f8014ab7791d9cca
Jul 07 18:30:30.970918 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:30.970879    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9" event={"ID":"8990248b-308e-48c4-ba3c-b956abf4ce32","Type":"ContainerStarted","Data":"1a593239cc8fa2a7552ca90be9db604dea25426806497b1b4b791856f0518ab3"}
Jul 07 18:30:30.971335 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:30.970951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9" event={"ID":"8990248b-308e-48c4-ba3c-b956abf4ce32","Type":"ContainerStarted","Data":"64efb5fbfa1849f63f29d16f0c0102cfb6563f216d1dec93f8014ab7791d9cca"}
Jul 07 18:30:30.990530 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:30.990472    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9" podStartSLOduration=1.990454555 podStartE2EDuration="1.990454555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:30:30.988563967 +0000 UTC m=+2192.878924694" watchObservedRunningTime="2026-07-07 18:30:30.990454555 +0000 UTC m=+2192.880815300"
Jul 07 18:30:39.993257 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:39.993216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:39.993679 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:39.993301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:40.006269 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:40.006240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:30:40.037239 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:30:40.037211    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:32:24.587354 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:24.587318    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"]
Jul 07 18:32:39.609134 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.609074    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9" podUID="8990248b-308e-48c4-ba3c-b956abf4ce32" containerName="main" containerID="cri-o://1a593239cc8fa2a7552ca90be9db604dea25426806497b1b4b791856f0518ab3" gracePeriod=45
Jul 07 18:32:39.748705 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.748674    2580 generic.go:356] "Generic (PLEG): container finished" podID="8990248b-308e-48c4-ba3c-b956abf4ce32" containerID="1a593239cc8fa2a7552ca90be9db604dea25426806497b1b4b791856f0518ab3" exitCode=0
Jul 07 18:32:39.748866 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.748744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9" event={"ID":"8990248b-308e-48c4-ba3c-b956abf4ce32","Type":"ContainerDied","Data":"1a593239cc8fa2a7552ca90be9db604dea25426806497b1b4b791856f0518ab3"}
Jul 07 18:32:39.876334 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.876308    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:32:39.947015 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.946979    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-dshm\") pod \"8990248b-308e-48c4-ba3c-b956abf4ce32\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") "
Jul 07 18:32:39.947204 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947042    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-tmp-dir\") pod \"8990248b-308e-48c4-ba3c-b956abf4ce32\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") "
Jul 07 18:32:39.947204 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxnsp\" (UniqueName: \"kubernetes.io/projected/8990248b-308e-48c4-ba3c-b956abf4ce32-kube-api-access-vxnsp\") pod \"8990248b-308e-48c4-ba3c-b956abf4ce32\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") "
Jul 07 18:32:39.947204 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-model-cache\") pod \"8990248b-308e-48c4-ba3c-b956abf4ce32\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") "
Jul 07 18:32:39.947204 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8990248b-308e-48c4-ba3c-b956abf4ce32-tls-certs\") pod \"8990248b-308e-48c4-ba3c-b956abf4ce32\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") "
Jul 07 18:32:39.947204 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-home\") pod \"8990248b-308e-48c4-ba3c-b956abf4ce32\" (UID: \"8990248b-308e-48c4-ba3c-b956abf4ce32\") "
Jul 07 18:32:39.947479 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947388    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8990248b-308e-48c4-ba3c-b956abf4ce32" (UID: "8990248b-308e-48c4-ba3c-b956abf4ce32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:32:39.947479 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947417    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-model-cache" (OuterVolumeSpecName: "model-cache") pod "8990248b-308e-48c4-ba3c-b956abf4ce32" (UID: "8990248b-308e-48c4-ba3c-b956abf4ce32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:32:39.947567 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.947532    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-home" (OuterVolumeSpecName: "home") pod "8990248b-308e-48c4-ba3c-b956abf4ce32" (UID: "8990248b-308e-48c4-ba3c-b956abf4ce32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:32:39.949498 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.949471    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8990248b-308e-48c4-ba3c-b956abf4ce32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8990248b-308e-48c4-ba3c-b956abf4ce32" (UID: "8990248b-308e-48c4-ba3c-b956abf4ce32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 18:32:39.949595 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.949477    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-dshm" (OuterVolumeSpecName: "dshm") pod "8990248b-308e-48c4-ba3c-b956abf4ce32" (UID: "8990248b-308e-48c4-ba3c-b956abf4ce32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 18:32:39.949595 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:39.949500    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8990248b-308e-48c4-ba3c-b956abf4ce32-kube-api-access-vxnsp" (OuterVolumeSpecName: "kube-api-access-vxnsp") pod "8990248b-308e-48c4-ba3c-b956abf4ce32" (UID: "8990248b-308e-48c4-ba3c-b956abf4ce32"). InnerVolumeSpecName "kube-api-access-vxnsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 18:32:40.048111 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.048058    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-tmp-dir\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:32:40.048111 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.048106    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxnsp\" (UniqueName: \"kubernetes.io/projected/8990248b-308e-48c4-ba3c-b956abf4ce32-kube-api-access-vxnsp\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:32:40.048111 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.048117    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-model-cache\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:32:40.048111 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.048127    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8990248b-308e-48c4-ba3c-b956abf4ce32-tls-certs\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:32:40.048111 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.048135    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-home\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:32:40.048434 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.048143    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8990248b-308e-48c4-ba3c-b956abf4ce32-dshm\") on node \"ip-10-0-134-243.ec2.internal\" DevicePath \"\""
Jul 07 18:32:40.756432 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.756407    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"
Jul 07 18:32:40.756845 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.756438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9" event={"ID":"8990248b-308e-48c4-ba3c-b956abf4ce32","Type":"ContainerDied","Data":"64efb5fbfa1849f63f29d16f0c0102cfb6563f216d1dec93f8014ab7791d9cca"}
Jul 07 18:32:40.756845 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.756490    2580 scope.go:117] "RemoveContainer" containerID="1a593239cc8fa2a7552ca90be9db604dea25426806497b1b4b791856f0518ab3"
Jul 07 18:32:40.779942 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.779889    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"]
Jul 07 18:32:40.783054 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:40.783029    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-200b05/auth-test-model-200b05-kserve-68958f45dc-wpxp9"]
Jul 07 18:32:42.596394 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:32:42.596355    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8990248b-308e-48c4-ba3c-b956abf4ce32" path="/var/lib/kubelet/pods/8990248b-308e-48c4-ba3c-b956abf4ce32/volumes"
Jul 07 18:36:23.713358 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:23.713321    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84d55d9f9f-mrfp7_a1d03d2a-0bf7-495e-bf44-49e1d17125d9/authorino/0.log"
Jul 07 18:36:29.761879 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:29.761839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8225m_9c174176-3864-4d15-8845-8b1327107b23/manager/0.log"
Jul 07 18:36:29.890813 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:29.890779    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sccnv_bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6/manager/0.log"
Jul 07 18:36:30.028356 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:30.028273    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-66f9f5f9bc-fxh7n_7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b/maas-api/0.log"
Jul 07 18:36:30.170412 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:30.170385    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724135-d579w_72011f40-2da4-419d-a639-08c3c3b4e163/cleanup/0.log"
Jul 07 18:36:30.291540 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:30.291461    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724150-bpcq6_a93c14c1-f18f-4cae-86f8-78f2382acb20/cleanup/0.log"
Jul 07 18:36:30.430725 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:30.430689    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5594d854cf-xt256_8f76511b-6c49-487d-b72d-e03e53b4ce1d/manager/0.log"
Jul 07 18:36:30.561771 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:30.561699    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pm4lx_2dc34b09-2df4-409a-add8-bae0cfb178d8/server/0.log"
Jul 07 18:36:30.689022 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:30.688990    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wmjsb_f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74/manager/0.log"
Jul 07 18:36:31.223275 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:31.223248    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cd65fb98-qxjkr_f0727af2-b2e2-4f4b-a35f-8186d4472b82/manager/0.log"
Jul 07 18:36:31.252887 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:31.252859    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cd65fb98-qxjkr_f0727af2-b2e2-4f4b-a35f-8186d4472b82/copy-manifests/0.log"
Jul 07 18:36:31.389479 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:31.389447    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-tspjk_7d39e3d3-eb55-4174-8b36-93ed62758f73/postgres/0.log"
Jul 07 18:36:32.286004 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.285973    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn_8c4a69c2-9565-4d59-9b5b-396b599671b4/util/0.log"
Jul 07 18:36:32.299230 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.299201    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn_8c4a69c2-9565-4d59-9b5b-396b599671b4/pull/0.log"
Jul 07 18:36:32.315369 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.315346    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn_8c4a69c2-9565-4d59-9b5b-396b599671b4/extract/0.log"
Jul 07 18:36:32.480159 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.480133    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj_318a0eb3-c407-4bae-9cd3-5ac4d511ced8/extract/0.log"
Jul 07 18:36:32.494569 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.494546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj_318a0eb3-c407-4bae-9cd3-5ac4d511ced8/util/0.log"
Jul 07 18:36:32.506788 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.506764    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj_318a0eb3-c407-4bae-9cd3-5ac4d511ced8/pull/0.log"
Jul 07 18:36:32.629461 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.629429    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss_5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f/util/0.log"
Jul 07 18:36:32.641347 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.641319    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss_5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f/pull/0.log"
Jul 07 18:36:32.652593 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.652567    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss_5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f/extract/0.log"
Jul 07 18:36:32.791405 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.791371    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27_ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0/util/0.log"
Jul 07 18:36:32.809851 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.809827    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27_ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0/pull/0.log"
Jul 07 18:36:32.825082 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.825052    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27_ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0/extract/0.log"
Jul 07 18:36:32.956337 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:32.956250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84d55d9f9f-mrfp7_a1d03d2a-0bf7-495e-bf44-49e1d17125d9/authorino/0.log"
Jul 07 18:36:33.089712 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:33.089681    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tgcjv_cb18c0ea-6a05-4439-b720-aeb19ba0f3bb/manager/0.log"
Jul 07 18:36:33.220413 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:33.220333    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mpg7w_17d61ef7-9e59-4c13-813e-f3fe2a5b685b/manager/0.log"
Jul 07 18:36:33.491945 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:33.491847    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ct58d_ada81ba7-7c02-4d96-88a7-7d1c3527e425/registry-server/0.log"
Jul 07 18:36:33.988223 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:33.988188    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hmwb2_f8f6024a-2301-4e0f-91df-9ee50965107f/manager/0.log"
Jul 07 18:36:34.840505 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:34.840475    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74ffcdc9c9-w7z4z_ec135987-68d8-4691-9177-675d3f6bf693/kube-auth-proxy/0.log"
Jul 07 18:36:34.982722 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:34.982691    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/1.log"
Jul 07 18:36:35.140580 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:35.140549    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kkxss_bebf4b7a-afe6-438c-8861-8ee027bf506b/payload-pre-processing/0.log"
Jul 07 18:36:35.286543 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:35.286505    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5s7q8_c1186ff2-608f-40c6-8a19-50eee9cfbf78/payload-processing/0.log"
Jul 07 18:36:35.843614 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:35.843579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-frcqg_b39da832-7f53-4b8d-ae4b-d0564361f0f6/main/0.log"
Jul 07 18:36:35.978788 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:35.978749    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-kjgq9_e356bdf5-5f2c-45f3-a0af-d8c78ccbd87f/main/0.log"
Jul 07 18:36:36.384114 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:36.384083    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-kkdxm_36122574-48f7-4d76-a678-783a3b20c141/main/0.log"
Jul 07 18:36:44.244439 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:44.244405    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mslq2_0e0f0ec3-356d-4bdd-b3e3-d434cb600b7b/global-pull-secret-syncer/0.log"
Jul 07 18:36:44.654180 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:44.654144    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8gqxt_eca02787-2090-4af2-a5f9-4a9484e386fc/konnectivity-agent/0.log"
Jul 07 18:36:44.800025 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:44.799991    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-243.ec2.internal_d0953349d49bbfb3ea71d025821a8f5d/haproxy/0.log"
Jul 07 18:36:48.209963 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.209927    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn_8c4a69c2-9565-4d59-9b5b-396b599671b4/extract/0.log"
Jul 07 18:36:48.267391 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.267360    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn_8c4a69c2-9565-4d59-9b5b-396b599671b4/util/0.log"
Jul 07 18:36:48.320883 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.320856    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pqfkn_8c4a69c2-9565-4d59-9b5b-396b599671b4/pull/0.log"
Jul 07 18:36:48.386092 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.386061    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj_318a0eb3-c407-4bae-9cd3-5ac4d511ced8/extract/0.log"
Jul 07 18:36:48.429449 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.429416    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj_318a0eb3-c407-4bae-9cd3-5ac4d511ced8/util/0.log"
Jul 07 18:36:48.476416 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.476334    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qnqwj_318a0eb3-c407-4bae-9cd3-5ac4d511ced8/pull/0.log"
Jul 07 18:36:48.533162 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.533129    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss_5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f/extract/0.log"
Jul 07 18:36:48.588934 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.588868    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss_5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f/util/0.log"
Jul 07 18:36:48.628472 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.628445    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73s4tss_5ffec9a6-dc28-4fbf-9fdb-75bcdd5e819f/pull/0.log"
Jul 07 18:36:48.681679 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.681648    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27_ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0/extract/0.log"
Jul 07 18:36:48.736983 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.736870    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27_ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0/util/0.log"
Jul 07 18:36:48.779147 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:48.779113    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1brf27_ebd9a90e-c5fc-46ee-92db-5c5a73cbe7d0/pull/0.log"
Jul 07 18:36:49.067013 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:49.066923    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84d55d9f9f-mrfp7_a1d03d2a-0bf7-495e-bf44-49e1d17125d9/authorino/0.log"
Jul 07 18:36:49.162040 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:49.162011    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-tgcjv_cb18c0ea-6a05-4439-b720-aeb19ba0f3bb/manager/0.log"
Jul 07 18:36:49.248166 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:49.248129    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mpg7w_17d61ef7-9e59-4c13-813e-f3fe2a5b685b/manager/0.log"
Jul 07 18:36:49.372108 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:49.372075    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ct58d_ada81ba7-7c02-4d96-88a7-7d1c3527e425/registry-server/0.log"
Jul 07 18:36:49.894636 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:49.894600    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hmwb2_f8f6024a-2301-4e0f-91df-9ee50965107f/manager/0.log"
Jul 07 18:36:52.592652 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:52.592625    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzlfz_c7138e6b-f922-40c2-a1e3-2e1fbac844c8/node-exporter/0.log"
Jul 07 18:36:52.627668 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:52.627635    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzlfz_c7138e6b-f922-40c2-a1e3-2e1fbac844c8/kube-rbac-proxy/0.log"
Jul 07 18:36:52.664721 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:52.664694    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzlfz_c7138e6b-f922-40c2-a1e3-2e1fbac844c8/init-textfile/0.log"
Jul 07 18:36:52.864824 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:52.864794    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-2bbsv_fd31438c-a233-4f6d-8b91-999917b80c6f/kube-rbac-proxy-main/0.log"
Jul 07 18:36:52.920368 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:52.920342    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-2bbsv_fd31438c-a233-4f6d-8b91-999917b80c6f/kube-rbac-proxy-self/0.log"
Jul 07 18:36:52.955841 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:52.955809    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-2bbsv_fd31438c-a233-4f6d-8b91-999917b80c6f/openshift-state-metrics/0.log"
Jul 07 18:36:53.446421 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.446386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-88djl_dc8f062a-97ef-40d9-81f4-9f6b7004d59f/prometheus-operator/0.log"
Jul 07 18:36:53.510166 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.510115    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-88djl_dc8f062a-97ef-40d9-81f4-9f6b7004d59f/kube-rbac-proxy/0.log"
Jul 07 18:36:53.539725 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.539688    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"]
Jul 07 18:36:53.540203 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.540179    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:36:53.540203 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.540201    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:36:53.540428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.540211    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8990248b-308e-48c4-ba3c-b956abf4ce32" containerName="main"
Jul 07 18:36:53.540428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.540219    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8990248b-308e-48c4-ba3c-b956abf4ce32" containerName="main"
Jul 07 18:36:53.540428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.540348    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4435bcc1-5cec-4843-80b4-0693cfd296f9" containerName="cleanup"
Jul 07 18:36:53.540428 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.540362    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8990248b-308e-48c4-ba3c-b956abf4ce32" containerName="main"
Jul 07 18:36:53.543661 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.543639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.551447 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.551429    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t248s\"/\"kube-root-ca.crt\""
Jul 07 18:36:53.551831 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.551815    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t248s\"/\"openshift-service-ca.crt\""
Jul 07 18:36:53.552485 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.552469    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t248s\"/\"default-dockercfg-ml2d6\""
Jul 07 18:36:53.581311 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.581282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"]
Jul 07 18:36:53.621811 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.621777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2fmf\" (UniqueName: \"kubernetes.io/projected/f43275e3-9b57-4499-a918-960d95620f0f-kube-api-access-w2fmf\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.622254 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.621827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-proc\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.622254 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.621933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-sys\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.622254 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.621990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-lib-modules\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.622254 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.622050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-podres\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.722745 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-sys\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.722745 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-lib-modules\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-podres\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2fmf\" (UniqueName: \"kubernetes.io/projected/f43275e3-9b57-4499-a918-960d95620f0f-kube-api-access-w2fmf\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722784    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-sys\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-proc\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722944    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-podres\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722945    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-lib-modules\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.723050 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.722986    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f43275e3-9b57-4499-a918-960d95620f0f-proc\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.736269 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.736232    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2fmf\" (UniqueName: \"kubernetes.io/projected/f43275e3-9b57-4499-a918-960d95620f0f-kube-api-access-w2fmf\") pod \"perf-node-gather-daemonset-67pn9\" (UID: \"f43275e3-9b57-4499-a918-960d95620f0f\") " pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.802880 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.802842    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-89d6cf79-6md5j_9655d5c2-690a-4e16-b9af-83a223e23aa3/telemeter-client/0.log"
Jul 07 18:36:53.840241 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.840212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-89d6cf79-6md5j_9655d5c2-690a-4e16-b9af-83a223e23aa3/reload/0.log"
Jul 07 18:36:53.853612 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.853585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:53.876154 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:53.876129    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-89d6cf79-6md5j_9655d5c2-690a-4e16-b9af-83a223e23aa3/kube-rbac-proxy/0.log"
Jul 07 18:36:54.005319 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:54.005286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"]
Jul 07 18:36:54.006269 ip-10-0-134-243 kubenswrapper[2580]: W0707 18:36:54.006243    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf43275e3_9b57_4499_a918_960d95620f0f.slice/crio-0e9f5aa87b5c50bc49946338264d33132b9e234c75ed305d991a553b072c5818 WatchSource:0}: Error finding container 0e9f5aa87b5c50bc49946338264d33132b9e234c75ed305d991a553b072c5818: Status 404 returned error can't find the container with id 0e9f5aa87b5c50bc49946338264d33132b9e234c75ed305d991a553b072c5818
Jul 07 18:36:54.263693 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:54.263594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9" event={"ID":"f43275e3-9b57-4499-a918-960d95620f0f","Type":"ContainerStarted","Data":"4d21f41d9c21074cfa21a103561d5e91b8ccb4d6a9d5b746b2ee130f68685ff1"}
Jul 07 18:36:54.263693 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:54.263645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9" event={"ID":"f43275e3-9b57-4499-a918-960d95620f0f","Type":"ContainerStarted","Data":"0e9f5aa87b5c50bc49946338264d33132b9e234c75ed305d991a553b072c5818"}
Jul 07 18:36:54.264099 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:54.263737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:36:55.279195 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:55.279158    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-j4l5g_6dd7a2cd-09e7-4a08-b5fd-ac1b1ca6c256/networking-console-plugin/0.log"
Jul 07 18:36:56.554483 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:56.554447    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f758b7bf7-m5k8w_01374afa-7bbb-44ff-81ba-eb1f21af8967/console/0.log"
Jul 07 18:36:56.616385 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:56.616353    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-fwc45_4bdb156a-5c6c-4901-a877-045041cf910d/download-server/0.log"
Jul 07 18:36:58.476134 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:58.476103    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-662vr_363e7a50-bf3c-4a89-bb95-8949a1f64c0b/dns/0.log"
Jul 07 18:36:58.516678 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:58.516651    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-662vr_363e7a50-bf3c-4a89-bb95-8949a1f64c0b/kube-rbac-proxy/0.log"
Jul 07 18:36:58.845877 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:58.845845    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ssnxv_b5f0f3c4-0c42-4df4-a9ad-614e98250cb8/dns-node-resolver/0.log"
Jul 07 18:36:58.901918 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:58.901866    2580 scope.go:117] "RemoveContainer" containerID="e8b426ed0bf6900c7392aa62191254717355a8a2ddf9d86a918b61802a4ea242"
Jul 07 18:36:59.533938 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:36:59.533877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j92kl_74f5befa-b459-42f7-806b-2eb48e2fd373/node-ca/0.log"
Jul 07 18:37:00.282994 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:00.282965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9"
Jul 07 18:37:00.308120 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:00.308070    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t248s/perf-node-gather-daemonset-67pn9" podStartSLOduration=7.308054434 podStartE2EDuration="7.308054434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:36:54.295380724 +0000 UTC m=+2576.185741448" watchObservedRunningTime="2026-07-07 18:37:00.308054434 +0000 UTC m=+2582.198415157"
Jul 07 18:37:00.981717 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:00.981693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74ffcdc9c9-w7z4z_ec135987-68d8-4691-9177-675d3f6bf693/kube-auth-proxy/0.log"
Jul 07 18:37:01.024718 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:01.024685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/0.log"
Jul 07 18:37:01.052325 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:01.052295    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/1.log"
Jul 07 18:37:01.100748 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:01.100715    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kkxss_bebf4b7a-afe6-438c-8861-8ee027bf506b/payload-pre-processing/0.log"
Jul 07 18:37:01.164957 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:01.164923    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5s7q8_c1186ff2-608f-40c6-8a19-50eee9cfbf78/payload-processing/0.log"
Jul 07 18:37:02.167806 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:02.167776    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c2z5b_01c155b4-3196-4c5e-9c49-c183312ffc68/serve-healthcheck-canary/0.log"
Jul 07 18:37:03.434943 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:03.434891    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v78vv_a9f6b888-066a-498a-afab-6da1a3ab8e9c/kube-rbac-proxy/0.log"
Jul 07 18:37:03.496392 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:03.496358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v78vv_a9f6b888-066a-498a-afab-6da1a3ab8e9c/exporter/0.log"
Jul 07 18:37:03.555978 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:03.555941    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v78vv_a9f6b888-066a-498a-afab-6da1a3ab8e9c/extractor/0.log"
Jul 07 18:37:04.643762 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:04.643736    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74ffcdc9c9-w7z4z_ec135987-68d8-4691-9177-675d3f6bf693/kube-auth-proxy/0.log"
Jul 07 18:37:04.680256 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:04.680226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/0.log"
Jul 07 18:37:04.708062 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:04.708033    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/1.log"
Jul 07 18:37:04.754944 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:04.754899    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kkxss_bebf4b7a-afe6-438c-8861-8ee027bf506b/payload-pre-processing/0.log"
Jul 07 18:37:04.803852 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:04.803815    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5s7q8_c1186ff2-608f-40c6-8a19-50eee9cfbf78/payload-processing/0.log"
Jul 07 18:37:06.307454 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.307422    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8225m_9c174176-3864-4d15-8845-8b1327107b23/manager/0.log"
Jul 07 18:37:06.369677 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.369638    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74ffcdc9c9-w7z4z_ec135987-68d8-4691-9177-675d3f6bf693/kube-auth-proxy/0.log"
Jul 07 18:37:06.402750 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.402712    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/0.log"
Jul 07 18:37:06.444760 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.444725    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-49h8f_422134db-9391-4170-a04a-613ea85b8ef0/istio-proxy/1.log"
Jul 07 18:37:06.480342 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.480298    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kkxss_bebf4b7a-afe6-438c-8861-8ee027bf506b/payload-pre-processing/0.log"
Jul 07 18:37:06.484080 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.484044    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sccnv_bfe1bb2a-7f13-431f-966e-0d1d65c3ddf6/manager/0.log"
Jul 07 18:37:06.547319 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.547285    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5s7q8_c1186ff2-608f-40c6-8a19-50eee9cfbf78/payload-processing/0.log"
Jul 07 18:37:06.547735 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.547722    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-66f9f5f9bc-fxh7n_7cbef1fe-4bf7-4ac0-a814-ce21aeb0990b/maas-api/0.log"
Jul 07 18:37:06.577785 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.577712    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724135-d579w_72011f40-2da4-419d-a639-08c3c3b4e163/cleanup/0.log"
Jul 07 18:37:06.615638 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.615606    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724150-bpcq6_a93c14c1-f18f-4cae-86f8-78f2382acb20/cleanup/0.log"
Jul 07 18:37:06.732788 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.732751    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5594d854cf-xt256_8f76511b-6c49-487d-b72d-e03e53b4ce1d/manager/0.log"
Jul 07 18:37:06.811666 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.811635    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pm4lx_2dc34b09-2df4-409a-add8-bae0cfb178d8/server/0.log"
Jul 07 18:37:06.886867 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:06.886827    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wmjsb_f3f9825b-d98b-4fb0-a3c1-b0d95a4a6a74/manager/0.log"
Jul 07 18:37:07.101486 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:07.101452    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cd65fb98-qxjkr_f0727af2-b2e2-4f4b-a35f-8186d4472b82/manager/0.log"
Jul 07 18:37:07.133809 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:07.133778    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76cd65fb98-qxjkr_f0727af2-b2e2-4f4b-a35f-8186d4472b82/copy-manifests/0.log"
Jul 07 18:37:07.182378 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:07.182290    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-tspjk_7d39e3d3-eb55-4174-8b36-93ed62758f73/postgres/0.log"
Jul 07 18:37:08.782729 ip-10-0-134-243 kubenswrapper[2580]: I0707 18:37:08.782697    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5p9kx_168475b4-ba86-490f-99d7-5816faa38363/openshift-lws-operator/0.log"